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Executive Summary 


Limited visibility is the single most critical factor affecting both the safety and capacity of worldwide 
aviation operations. In commercial aviation alone, over 30-percent of all fatal accidents worldwide are 
categorized as controlled flight into terrain (CFIT), where a mechanically sound and normally functioning 
airplane is inadvertently flown into the ground, water, or an obstacle, principally due to the lack of outside 
visual reference and situation awareness. The National Aeronautics and Space Administration (NASA) 
Aviation Safety Program’s Synthetic Vision Systems (SVS) Project is developing technologies with 
practical applications that will mitigate low visibility conditions as a causal factor to civil aircraft 
accidents, as well as replicate the operational benefits of flight operations in unlimited ceiling and 
visibility day conditions, regardless of the actual outside weather or lighting condition. The technologies 
will emphasize the cost-effective use of synthetic/enhanced- vision displays; worldwide navigation, 
terrain, obstruction, and airport databases; and Global Positioning System (GPS)-derived navigation to 
mitigate “visibility-induced” (lack of visibility) errors for all aircraft categories. A major thrust of the 
SVS Project is to develop and demonstrate affordable, certifiable display configurations which provide 
intuitive out-the-window terrain and obstacle information, including guidance information for precision 
navigation and obstacle/obstruction avoidance for commercial and business aircraft. 

To date, much of the SVS research has focused on introducing SVS display technology into as many 
existing aircraft as possible by providing a retrofit approach. This approach employs existing head down 
display (HDD) capabilities for glass cockpits (cockpits already equipped with raster-capable HDDs) and 
head-up display (HUD) capabilities for the other aircraft. A major NASA flight test at Dallas/Fort Worth 
(DFW) airport and several simulator studies have occurred for assessment and evaluation of the SVS 
developments and the retrofit approach. The HDD objective of these studies was to examine whether an 
SVS display could be retrofitted into an Electronic Flight Instrumentation System (EFIS) Size “A” (5.25 
in. wide by 5 in. tall) (e.g., B-757-200) Electronic Attitude Direction Indicator (EADI) and Size “D” (6.4 
in. wide by 6.4 in. tall) (e.g., B-777) Primary Flight Display (PFD). A Size “X” (9 in. wide by 8 in. tall) 
head-down display was also tested that may represent the display real estate available on future aircraft. 
The HUD objective was to examine the feasibility of the concept of retrofitting SVS display technology 
with HUDs for aircraft without raster-capable HDDs. The feasibility of the concept of retrofitting SVS 
display technology with HUDs was verified for nighttime operations. Two terrain-texturing techniques 
were also evaluated during the research. One method of terrain texturing, generic texturing, involved the 
selection of terrain color based on absolute altitude. The other method of terrain texturing, photo-realistic 
texturing, employed full-color ortho-rectified aerial photographs draped over the elevation model. The 
results of those studies confirmed that an SVS display, with pilot-selectable field of view (FOV), could be 
incorporated as part of an EFIS suite and effectively replace an EADI or PFD. Regardless of HDD 
display size, and for the nighttime HUD application, pilots reported greater situation awareness and had 
lower flight technical error (FTE) while operating with the SVS displays compared to conventional 
displays. For both HDD and HUD applications, no significant performance effects were found between 
texturing techniques, although most of the pilots preferred the photo-realistic terrain texturing technique 
to the generic texturing technique. 

The results of previous research have documented the unquestionable promise of SVS to enhance 
situation awareness and improve aviation safety during approaches to terrain and operationally-complex 
airports. The DFW flight test showed that ah SVS display concepts provided precise path control and 
significantly improved spatial awareness for approaches to the DFW airport under nighttime conditions. 
Although the fixed-based simulator results had provided convincing data on the efficacy of SVS for 
terrain-challenged environments, these results had yet to be replicated and validated under operational 
conditions like that conducted at DFW in 2000. In 1999, a flight test / demonstration was conducted 
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using the SVS technology to make approaches to the Asheville, North Carolina (AVL) airport, but no 
empirical data was collected to substantiate the claims that SVS was effective in terrain-challenged 
environments. Therefore, the Aviation Safety program and SVS Project conducted research flight 
operations at Eagle-Vail, CO (EGE) to further examine the utility, capabilities, and potential of SVS to 
enhance situation awareness and improve pilot performance for complex approaches in mountainous 
environments. 

The flight test was conducted to evaluate three SVS display types (Head-up Display, Head-Down Size 
A; Head-Down Size X) and two terrain texture methods (photo-realistic, generic) in comparison to the 
simulated Baseline Boeing-757 Electronic Attitude Direction Indicator and Navigation / Terrain 
Awareness and Warning System displays. These independent variables were evaluated for path error, 
situation awareness, and workload while making approaches to Runway 25 and 07 and during simulated 
engine-out Cottonwood 2 and KREMM departures. The results of the experiment showed significantly 
improved performance, situation awareness, and workload for SVS concepts compared to the Baseline 
displays and confirmed the retrofit capability of the Head-Up Display and Size A SVS concepts. The 
research also demonstrated that the tunnel guidance display concept used within the SVS concepts 
achieved required navigation performance (RNP) criteria. These findings are a strong verification of the 
SVS retrofit approach. That is, HUD or HDDs of any size or texture method tested were an equally 
effective means of implementing SVS concepts to achieve FTE and RNP benefits. The top-level results 
of the EGE flight test concerning the improved path performance, enhanced situation awareness, and 
lower associated workload provided by all of the SVS (HDD and HUD) concepts, regardless of display 
size, are highly significant. These results firmly establish the SVS retrofit concept approach as viable. 

This paper documents the EGE flight test experiment and presents suggestions for research thrusts for 
further development of future embodiments of synthetic vision systems. 

Introduction 

The Synthetic Vision Systems (SVS) element of the National Aeronautics and Space Administration’s 
(NASA) Aviation Safety Program (AvSP) is striving to eliminate poor visibility as a causal factor in 
aircraft accidents as well as enhance operational capabilities of all aircraft. To accomplish these safety 
and situation awareness (SA) improvements, the SVS concept is designed to provide the pilot an 
unobstructed view of the world around the aircraft through the display of computer-generated imagery 
derived from an onboard database of terrain, obstacle, and airport information. To accomplish the 
operational enhancements, the SVS concept includes the intuitive display of intended flight path by tunnel 
or pathway-in-the-sky presentations. When coupled with a synthetic or enhanced view of the outside 
world, the spatially-integrated depiction of the intended aircraft flight path and its relation to the world 
provides an intuitive, easily interpretable display of flight-critical information for the pilot. 

The ability of a pilot to ascertain critical information through visual perception of the outside 
environment can be limited by various weather phenomena, such as rain, fog, and snow. Since the 
beginning of flight, the aviation industry has continuously developed various devices to overcome low- 
visibility issues, such as attitude indicators, radio navigation, and instrument landing systems. Recent 
advances include moving map displays, incorporating advances in navigational accuracies from the 
Global Positioning System (GPS), and Terrain Awareness and Warning Systems (TAWS), such as 
Honeywell’s Enhanced Ground Proximity Warning System (EGPWS). All of the aircraft information 
display concepts developed to date, however, require the pilot to perform various additional levels of 
mental model development, maintenance, and information decoding in a real-time environment when 
outside visibility is restricted (e.g., Theunissen, 1997). 
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Better pilot SA during low visibility conditions is potentially offered by SVS displays because of the 
natural cues provided by a three-dimensional perspective display of the outside world showing unlimited 
ceiling and visibility conditions. New technological developments in navigation performance, low-cost 
attitude and heading reference systems, computational capabilities, and display hardware allow for the 
prospect of SVS displays for virtually all aircraft classes. SVS display concepts employ computer- 
generated terrain imagery, on-board databases, and precise position and navigational accuracy to create a 
three dimensional perspective presentation of the outside world, with necessary and sufficient information 
and realism, to enable operations equivalent to those of unlimited ceiling and visibility conditions 
regardless of the actual outside weather. The safety outcome of SVS is a display that should help reduce, 
or even prevent, controlled flight into terrain (CFIT), which is the single greatest contributing factor to 
fatal worldwide airline and general aviation accidents (Boeing, 1998). Other safety benefits include 
reduced runway incursions and loss-of-control accidents (Williams et al., 2001). Operational benefits 
potentially include more approach and departure options and lower visibility minimums for SVS 
equipped aircraft. For a more detailed description of the SVS project, please see Appendix A. 

Safety Benefits of SVS 

It is highly unlikely that conventional display concepts can significantly increase safety as new 
functionality and new technology cannot simply be layered onto previous design concepts since the 
current system complexities are already too high (Theunissen, 1997). Better human- machine interfaces 
require a fundamentally new approach. One such approach applies the fundamental advantage of 
perspective flight path displays relative to conventional displays. Rather than commanding the pilot what 
to do, or at best showing only the error with respect to the desired trajectory, guidance and navigation 
displays can now provide information about the margins within which the pilot is allowed to operate. 
These displays are augmented to show such information as spatial constraints and terrain constraints, 
rather than just showing conventional flight director commands, which only indicate an error, seemingly 
independent of the actual constraints. These additional display elements provide guidance that does not 
force the pilot to apply a continuous compensatory control strategy. Only in this way can human 
flexibility be exploited. This is a fundamental difference between current and SVS displays - that 
synthetic vision embodies the concept of “human-centered” design by providing natural versus coded 
information to the pilot (Theunissen, 1997). Appendix B describes the concept of human-centered design 
and the theoretical foundations for synthetic vision. 

Because SVS displays are posited to provide both natural and coded information to the pilot, better 
pilot spatial situation awareness during low visibility conditions can be achieved. Synthetic vision 
technology will allow the issues associated with limited visibility to be solved with a vision-based 
solution, making every flight the equivalent of a clear daylight operation, which will help improve 
situation awareness, lower workload, and support proper development of pilots’ mental models (see 
Appendix C). Therefore, SVS can have a most significant impact on improving aviation safety, as limited 
visibility has often been cited as the single greatest contributing factor in fatal worldwide airline and 
general aviation accidents (Boeing, 1996). 

Consider that one of the major classifications of aviation accidents involving visibility issues is CFIT 
and that CFIT is the greatest cause of aviation fatalities. A CFIT accident is defined as “one in which an 
otherwise-serviceable aircraft, under control of the crew, is flown (unintentionally) into terrain, obstacles 
or water, with no prior awareness on the part of the crew of the impending collision” (Wiener, 1977). 
Enders, et al., (1996) noted that worldwide, the chances of CFIT accidents are 5 times higher in non- 
precision approaches. SVS could have a significant impact in ameliorating the incidence of this accident 
category (Arthur et al., 2003). For commercial transport aircraft, instant recognition and correction of 
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visibility-induced errors may eliminate CFIT. If accurate positioning information of other traffic were 
incorporated into the system, SVS could also help to eliminate runway incursions. For general aviation 
aircraft, a lower cost implementation of such a system could help to prevent visibility-induced loss-of- 
control accidents by providing an intuitive, easy-to-fly visual reference for Visual Meteorological 
Conditions (VMC)-like operations in Instrument Meteorological Conditions (IMC). It would also be 
anticipated that SVS technology could serve to increase national airspace system capacity by providing 
the potential for VMC-like operations more of the time (Hemm, 2000; Hemm, Lee, Stouffer, & Gardner, 
2001 ). 

Operational Benefits of SVS 

Aside from the safety benefits accrued from the increased SA with reduced workload provided by the 
natural information presentation of synthetic vision displays, considerable operational benefits are also 
potentially available through provisions for flight operations in IMC resembling those conducted in 
VMC. These potential benefits could include lower landing minimums, more approach and departure 
options, more complex path structures to avoid hazardous or restricted (noise, security) areas, reduced 
training time, etc. Among these, a significant operational benefit of SVS would include helping to meet 
new Federal Aviation Administration (FAA) required navigation performance (RNP) criteria. 

RNP is a statement of the navigation performance accuracy necessary for operation within a defined 
airspace. RNP airspace is a generic term referring to airspace, routes, and legs, where minimum 
navigation performance requirements have been established and aircraft must meet or exceed that 
performance to fly in that airspace. The system performance requirements for RNP Area Navigation 
(RNAV) is that each aircraft operating in RNP airspace shall have total horizontal system error 
components in the cross-track and along-track directions that are less than the RNP value 95% of the 
flying time. Vertical navigation (VNAV) capability further enhances flight operations by enabling the 
specification of a flight path vertically for the lateral flight path. The system performance requirements 
for VNAV are that for at least 99.7% of the time the navigational performance in the vertical plane, or the 
total vertical system error, is less than a specified altitude deviation measure based on the airspace being 
flown in (below 5000 feet mean sea level (MSL), 5000-10000 feet MSL, above 10000 feet MSL) and the 
type of flight operation (level flight/climb/descent or flight along specified vertical profile) being 
performed (see table 1). For more information about RNP, please see Appendix D. 


Table 1. Vertical Accuracy Performance Requirements 


Error Source 

Level Flight Segments 
and Climb/Descent 
Intercept of Clearance 
Altitudes (MSL) 

Approach 
along Specified 
Vertical Profile 
(MSL) 


At or Below 
5000 ft 

5000 ft to 
10000 ft 

Above 
10000 ft 

At or Below 
5000 ft 

5000 ft to 
10000 ft 

Above 10000 ft 

Altimetry 

90 ft 

200 ft 

250 ft 

140 ft 

265 ft 

350 ft 

RNAV 

Equipment 

50 ft 

50 ft 

50 ft 

100 ft 

150 ft 

220 ft 

Flight 

Technical 

150 ft 

240 ft 

240 ft 

200 ft 

300 ft 

300 ft 

Total Root- 

Sum-Square 

(RSS) 

190 ft 

320 ft 

350 ft 

265 ft 

430 ft 

510 ft 
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RNP defines that an aircraft’s actual navigation performance will be required to meet certain 
navigation precision criteria. Synthetic Vision is postulated to contribute significantly to achieving RNP 
through the use of pathway displays and guidance symbology. Research has shown that the use of 
pathway displays and predictive guidance can significantly reduce flight technical error (e.g., Haskell & 
Wickens, 1993; Wickens & Prevett, 1995; Theunissen, 1997). Therefore, SVS may not only increase 
aviation safety but may also have significant operational benefits that, in turn, can result in substantial 
economic benefits. 

Economic Benefits of SVS 

Hemm (2000) and Hemm et al. (2001) did a benefit estimation of SVS technologies and 
concluded that synthetic vision has significant potential not only for improving aviation safety and 
increasing navigation precision but can also provide for other considerable economic benefits. The 
analyses were based on the assumptions that SVS could reduce runway occupancy time in low visibility; 
reduce departure minimums; reduce arrival minimums; better allow for converging and circling 
approaches, especially for dual and triple runway configurations; reduce inter-arrival separations; and 
provide for independent operations on closely-spaced parallel runways. A cost-benefit analysis, based on 
those assumptions at 10 airports (DFW, ORD, LAX, ATL, DTW, MSP, EWR, SEA, LGA, JFK), 
calculated the average cost savings to airlines for the years 2006 to 2015 to be $2.25 billion. Part of the 
rationale for the AvSP SVS Project choosing Dallas/Fort Worth (DFW) (Glaab et al., 2003) and Eagle 
County, Colorado Regional (EGE) airport (Kramer et al., 2003) locations to conduct flight test research 
was to demonstrate that SVS could allow operations into runways (i.e., EGE Runway 07) that normally 
are not used, especially in IMC. 

Challenges to Synthetic Vision 

Although there is significant potential for SVS to help meet national aviation goals, there are still 
considerable research challenges that need to be addressed. To identify these challenges, a workshop 
resulting in a concept of operations for commercial and business aircraft was held in early 2000 
(Williams, et al., 2001). A similar workshop was hosted that focused on general aviation aircraft. The 
focus of these events was to obtain wide ranging input on the benefits and features which synthetic vision 
might incorporate. This meeting included representatives from NASA, Department of Defense, FAA, 
industry, professional organizations, airlines, aircraft and avionics manufacturers, airports, and academic 
institutions. The result of the workshop was a “shopping list” of research issues that need to be explored 
in developing SVS display concepts. Primary among them, expressed as questions, are: 

• How can an SVS display be retrofitted into an aircraft class that has limited real-estate display 
space? 

• What are the perceptual issues involved with ramification and increased field of views on 
smaller SVS displays? 

• What is the best way to present synthetic terrain and symbology to enhance situation 
awareness? 

• Can an SVS display improve SA and reduce workload? 

• What are the cognitive issues that may affect safety of flight when pilots fly a “compelling” 
synthetic vision scene? 
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• What is the best way to display the synthetic vision scene and what are the database and 
sensor requirements to ensure the integrity of a synthetic vision system? 

The NASA SVS-Commercial and Business aircraft element is using these issues as guidance in its 
research plans and a significant amount of research has been conducted, as discussed in the next section. 

NASA SVS Human Factors Research 

NASA SVS Simulator Research Experiments. An important issue for the SVS concept is whether 
useful and effective SVS displays can be implemented on limited size display spaces as would be required 
to implement this technology on older aircraft with physically smaller instrument spaces. With computer 
generated 3-dimensional imagery, SVS display concepts can provide pilot-selectable display Field of 
View (FOV) control to enhance display effectiveness, potentially overcoming size constraint limitations. 
Comstock, et al., (2001) conducted a study to examine how approaches using SVS on smaller display 
sizes affect performance and situation awareness. In this study, prototype SVS displays were shown on 
the following display sizes: (a) ARINC Size A (e.g., 757 Electronic Attitude Direction Indicator), (b) 
ARINC Size D (e.g., 777 Primary Flight Display), and (c) new size “X” (Rectangular flat-panel, 
approximately 20 x 25 cm). Testing was conducted in the Visual Imaging Simulator for Transport 
Aircraft Systems (VISTAS)-I, which is a high-resolution graphics workstation at NASA Langley 
Research Center. Specific issues under test included the display size and the FOV to be shown on the 
display and, directly related to FOV, the degree of minification of the displayed image or picture. Using 
simulated approaches to Asheville, NC airport runways (FAA identifier, AVL), no significant lateral or 
vertical performance differences were found for any display size or FOV condition. Preferred FOV based 
on performance was determined by using approaches during which pilots could select FOV. Mean 
preference ratings for FOV were in the following order: (1) 30°, (2) Unity, (3) 60°, and (4) 90°, and held 
true for all display sizes tested. 

A second study by Stark, et al., (2001) was conducted to further explore issues of display size, FOV, 
and tunnel guidance on pilot performance and SA. Tunnel guidance was expected to improve pilot 
performance, increase SA, and decrease workload. Results of the study supported these hypotheses. 
Both horizontal and vertical path error were indeed reduced by tunnel guidance. Pilots were able to stay 
on path more accurately with the assistance of the tunnel, and these findings support previous pathway 
research (see Doherty & Wickens, 2001 for a review). Pilots also verbally reported “feeling” like they 
were better able to stay on path when using the tunnel guidance system, and these feelings were 
accompanied by a statistically significant reduction in mental workload measurements. One pilot 
commented that he was so “in tune” with the guidance system that it was almost “too easy” to fly. The 
“intuitiveness” of the tunnel concept contributed to significantly higher situation awareness ratings for the 
synthetic vision concepts. 

Dallas Fort-Worth Flight Test. To introduce SVS display technology into as many existing aircraft 
as possible, a retrofit approach was postulated. That approach proposed using existing head-down display 
(HDD) capabilities for glass cockpits (cockpits already equipped with raster-capable HDDs) and head-up 
display (HUD) capabilities for the other aircraft. This retrofit approach takes advantage of the growing 
numbers of HUDs being fitted into the commercial fleet due to HUD operational benefits. 

Previous research in a fixed-based simulator at NASA Langley Research Center (LaRC) (Comstock et 
al., 2001) indicated that an SVS display could significantly enhance SA in both an operationally 
challenging environment (multiple runways and taxiways at DLW) and a terrain-challenged environment 
(at AVL). The SVS retrofit approach was evaluated and initially validated for typical nighttime airline 
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operations at DFW International Airport in September 2000. Overall, 6 evaluation pilots performed 75 
research approaches accumulating 18 hours of flight time evaluating SVS display concepts using the 
NASA Langley Research Center’s Airborne Research Integrated Experimental System (ARIES) Boeing 
B-757-200 aircraft. The SVS HDD concepts evaluated included variations in display size, with pilot- 
selectable FOV, and in methods of terrain texturing. SVS HUD concept evaluations also included 
variations in the method of terrain texturing. Two types of terrain texturing were employed: photo- 
realistic texturing and elevation-based color-coded texturing (also referred to as generic texturing). 

Results (Glaab et al., 2003) indicated that effective applications of SVS display technology can be 
accomplished in aircraft equipped with HDDs as small as Size A (5.25 in. wide by 5 in. tall) using pilot- 
selectable FOV. All pilots acknowledged the enhanced situation awareness provided by all of the SVS 
(HDD and HUD) concepts. Regardless of display size, pilots selected HDD FOVs of approximately 50 
degrees, or higher, during initial approach segments, such as on downwind and base legs, and consistently 
reduced the selected FOV to approximately 30 degrees, or less, for low final approach segments. Display 
size, selected FOV, and minification are correlated. Therefore, the selected FOV/phase-of-flight result 
above can be expressed in another way - as range to touchdown decreased, the minification factor moved 
toward unity (i.e., no minification). Also, pilots selected smaller minification factors for the larger-sized 
HDDs regardless of phase-of-flight (as display size increased, the minification factor moved toward 
unity). With these results, pilot-selectable display FOV control became the accepted standard approach to 
overcome display size constraint limitations associated with HDDs within the NASA SVS Project. 

The majority of the pilots participating in the DFW tests preferred the photo-realistic terrain texturing 
technique to the generic texturing technique for both HDD and HUD applications. For aircraft without 
raster-capable HDDs, the feasibility of the concept of retrofitting SVS display technology with HUDs was 
verified for nighttime operations by results demonstrating effective SVS presentation on this type of 
display device. Pilots also commented that presentation of SVS imagery on the HUD (with a minification 
factor of unity - i.e., no minification) was not only acceptable, but actually preferred, over the HDDs. The 
top-level results of the DFW flight test concerning the enhanced situation awareness provided by all of 
the SVS (HDD and HUD) concepts, regardless of display size, are highly significant. These results 
firmly established the SVS retrofit concept approach as viable, at least in the benign terrain environment 
of DFW in nighttime operations. 


Current Study 

The results of previous research have documented the unquestionable promise of SVS to enhance 
situation awareness and improve aviation safety during approaches to terrain- (e.g., 1999 AVL 
demonstration; Comstock et al., 2001; Stark et al., 2001) and operationally-complex (e.g., 2000 DFW 
flight test) airports. The DFW flight test showed that all SVS display concepts provided precise path 
control and significantly improved spatial awareness for approaches to the DFW airport under nighttime 
conditions. However, although the fixed-based simulator results had provided convincing data on the 
efficacy of SVS for terrain-challenged environments, these results had yet to be replicated and validated 
under operational conditions like that conducted at DFW in 2000. In 1999, a flight test / demonstration 
was conducted using the SVS technology to make approaches to AVL airport, but no empirical data was 
collected to substantiate the claims that SVS was effective in terrain-challenged environments. Therefore, 
the Aviation Safety Program and SVS Project conducted research flight operations at Vail, CO to further 
examine the utility, capabilities, and potential of SVS to enhance situation awareness and improve pilot 
performance for complex approaches in mountainous environments. This paper documents that flight test 
experiment. 
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Goals and Objectives 


The goal of the flight test conducted at EGE was to extend the assessment of the SVS retrofit approach 
to operations in a terrain-challenged operational environment with testing in daytime conditions. EGE 
represented an ideal location to test the effectiveness of SVS technologies for terrain awareness and 
separation for approaches and departures that put the aircraft close to mountainous terrain. 

SVS Display Concepts 

Six NASA SVS tactical display configurations were evaluated. These configurations were obtained 
by presenting SVS terrain databases of two terrain texturing types (generic or photo-realistic) on three 
displays: a HUD and two HDD sizes (A or X). 

Terrain Texturing Types. Terrain-texturing refers to the method used to fill the polygons that 
comprise the rendered terrain database. The two texturing concepts tested were elevation-based color- 
coding. or generic, and photo-realistic. The generic texturing concept consisted of applying equal-height 
coloring bands that correspond to different absolute terrain elevation levels, similar to the colors 
employed for Visual Flight Rules sectional charts. Lower terrain levels were colored with darker colors, 
while higher terrain levels were assigned lighter colors. A certain shade of green was set to the field 
elevation. The photo-realistic texturing was derived from full color ortho-rectified aerial photographs. 
The resulting scene was a highly realistic view due to the photographic imagery employed. 

HUD Concept. As mentioned previously, the NASA SVS Project is investigating the potential of 
HUD technology as a retrofit solution for display of terrain database SVS concepts in non-glass cockpits. 
As such, the HUD is used in a manner not traditionally employed in commercial aircraft operations. The 
SVS terrain database scene is presented on the HUD as a raster image with stroke symbology overlaid 
upon it. This concept for a SVS-HUD is similar to enhanced vision system (EVS) concepts, which 
typically use advanced imaging sensors to penetrate weather phenomena such as darkness, fog, haze, rain, 
and/or snow, and the resulting enhanced scene is presented on a HUD, through which the outside real 
world may be visible. (The FAA has just recently certified an infrared EVS for use on a business 
aircraft.) In the SVS-HUD concept, the terrain database scene is displayed instead of the sensor EVS 
image. Unlike EVS displays, the SVS-HUD concept maintains a “clear sky” so there is no obstruction of 
that area of the display. (This is in contrast to an EVS image which displays a sensor image of the sky.) 
Below the horizon, the terrain raster image can obstruct the view of the outside real world, just as an EVS 
display can, particularly if the raster brightness is not appropriately controlled by the pilot. Obstruction of 
the outside real world scene by such a display is a recognized certification issue and a rapid means of 
decluttering the SVS imagery from the HUD was provided to the evaluation pilot. In addition to the 
raster, nominal flight information symbology characteristic of most airline HUDs was overlaid on the 
HUD imagery. 

HDD Tactical Display Sizes. Two different SVS-HDD configurations were evaluated during this 
flight test to explore retrofit concepts of SVS display technology into existing glass cockpits (cockpits 
already equipped with raster-capable HDDs). One display configuration, referred to as the SVS Size A 
(5.25 in. wide by 5 in. tall), was similar to a B-757-200 electronic attitude direction indicator (EADI) with 
separate airspeed, altitude, and vertical speed gauges, with the addition of SVS information. The second 
HDD configuration, referred to as Size X (9 in. wide by 8 in. tall), featured an enlargement of an 
integrated Primary Flight Display (PFD) to replicate future SVS HDD concepts. Evaluation pilots could 
control the FOV of the HDD EADI and PFD concepts evaluated to enhance display effectiveness. A 
conventional Size A EADI HDD configuration with no SVS information was also provided as a baseline 
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for comparison purposes. Terrain information was available on the Navigation Display (ND) (4.75 in. 
wide by 6 in. tall) for all of the concepts. 

The objectives of the flight test were to: 

a) Confirm potential of NASA SVS HUD concept as a retrofit solution for display of SVS 
concepts in non-glass cockpits. Determine potential in both day VMC and in day, low- 
visibility operational environments. 

b) Confirm results from piloted simulation experiments and SVS-DFW flight test for operational 
utility and acceptability of various-sized (Size A, X) head-down synthetic vision displays. 

c) Compare operational utility and acceptability of photo-realistic textured with generic textured 
terrain databases within NASA SVS concepts (HUD; head-down Size A, X). 

d) Assess pilot path control performance (flight technical error) during manually flown landing 
approach and go-around maneuvers in a terrain-challenged operational environment, with and 
without SVS display concepts. 

e) Determine required navigation performance capabilities of SVS for area navigation. 

f) Confirm the situation awareness and workload benefits of SVS display concepts. 

g) Provide demonstration of economic potential of SVS for approaches that have significant 
restrictions for current operations. 


Hypotheses 

The a priori hypotheses (based on prior SVS research) for the flight test evaluations included: 

Objective Data Hypotheses 

a) All SVS display concepts would have lower flight technical error (FTE) compared to a 
Baseline EADI/ND. 

b) Pilots would have lower FTE with the SVS HUD concepts than with the Size X display 
concepts. 

c) Pilots would have lower FTE with the Size X displays than with the smaller Size A SVS 
displays. 

d) Pilots would have lower FTE with the photo-realistic texture concepts. 

Subjective Data Hypotheses 

e) Pilots would prefer all SVS display concepts over the Baseline EADI/ND. 

f) Pilots would prefer the HUD over the Size X display concepts. 

g) Pilots would prefer the Size X display concepts to the smaller Size A SVS display concepts. 

h) All SVS display concepts would be rated higher in situation awareness than the Baseline 
EADI/ND. 

i) Pilots would rate the Size X display concepts significantly higher in terms of situation 
awareness than the Size A SVS display concepts. 
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j) There would be an interaction found between display size and texture for situation awareness 
ratings. For example, photo-realistic texture would be judged to be much more effective than 
generic texturing on the HUD than on Size A, where texturing effects would be much less 
important. 

k) Pilots would rate the photo-realistic texture concept significantly higher in terms of situation 
awareness than the generic texture concept for both HDD and HUD display concepts. 

l) Workload ratings would be significantly lower for all SVS display concepts compared to the 
Baseline EADI/ND. 

m) Workload ratings would be significantly lower for the HUD compared to the Size X display 
concepts. 

n) Workload ratings would be significantly lower for the Size X compared to the Size A SVS 
display concepts. 

o) Workload ratings would be significantly lower for the Size A SVS display concepts compared 
to the Baseline EADI/ND. 


Method 


Subjects 

Six evaluation pilots, representing three airlines, FAA and Boeing, flew 12 research flights totaling 
51.6 flight hours. Eighty-four data flight test runs were conducted to evaluate the NASA display concepts 
with forty- nine being flown to Runway 07 and thirty-five flown to Runway 25. All participants were 
rated B-757 captains with operating experience at EGE. The currency and degree of actual experience at 
EGE prior to the test varied but was not considered to be a significant factor since, prior to deployment, 
all evaluation pilots received a one-day training course (briefing and simulator session) at NASA LaRC. 
This training was to familiarize them with the SVS display concepts as well as provide sufficient training 
in the approach procedures and tasks to create a consistent level of knowledge and experience for EGE 
operations. 

Simulator and Flight Test Vehicle 

Aircraft. The flight test was conducted in an operationally realistic, terrain-challenged 
environment using the NASA LaRC Boeing 757 (B-757) ARIES aircraft to compare SVS display 
concepts to a Baseline EADI and navigation display (ND) which included a simulated TAWS. The left 
seat in the Boeing 757 was occupied by the Evaluation Pilot (EP). This position, with its associated 
displays and controls, is used for research testing and is known as the Flight Deck Research System 
(FDRS). The right seat was occupied by a NASA Safety Pilot (SP). The left seat included the installation 
of an SVS Research Display (SVS-RD) and an overhead HUD projection unit (see fig. 1). A vision 
restriction device (VRD) (fig. 2) was placed in the left-seat forward windscreen to block the EP’s forward 
vision and thus simulate IMC when needed experimentally. 
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Figure 1. SVS-RD installed in ARIES 757 aircraft. 



Figure 2. Vision Restriction Device in ARIES 757 aircraft. 

SVS-RD. The SVS-RD was a Commercial Off-the-Shelf 18.1 in. diagonal high brightness 
Liquid Crystal Display monitor, modified for installation over the forward instrument panel 
cluster on the left hand side of the ARIES cockpit (see fig. 1). Since the SVS-RD was capable of 
generating high resolution, multi-sized displays, this monitor displayed all head-down display 
concepts for evaluation. The SVS-RD covered the normal Boeing 757 captain’s displays with 
the exception of the analog standby instruments (attitude direction indicator, airspeed, and 
altitude). The SVS-RD had 1280 vertical x 1024 horizontal pixel resolution (approximately 90 
pixels per inch) with 900 nits brightness for reasonable sunlight readability in the Boeing 757 
aircraft. Overall, the SVS-RD weighed approximately 16 lbs. 

Field of View Control. FOV control for the NASA HDD SVS concepts was available to the 
pilot on a four-position wafer switch on a conveniently located center console panel. The FOV options 
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were: Unity, 30°, 60°, and 90°. The FOV provided at the ’’unity” setting changed depending upon the size 
of the experimental HDD condition being flown. 

Synthetic Vision Systems Graphic Engine (SVS-GE). The NASA SVS display concepts were 
generated by two Intergraph ZxlO personal computers using Windows NT ™. The ZxlO used for this 
flight test was a dual 866 MHz Pentium® III processor with 1+ Gigabytes of Random Access Memory. 
The video card used was a 3D Labs, Inc. Wildcat™ 4210 which provided 1280x1024 resolution at 60 
Hertz (Hz) anti-aliased (SuperScene ™ enabled) video rendering at real-time (>30 Hz) update rates. 
Symbology was generated using the OpenGL application programming language. SXGA (1280x1024 
pixels, 60 Hz non-interlaced) format video output from the ZxlO computer drove the SVS-RD. For the 
HUD raster channel, an XGA (1024x768 pixels) image from the ZxlO was scan converted to RS-343 
(875 line, 30 Hz interlaced) format video via a Folsom 2100 scan conversion unit. 

Head-Up Display. The left-seat, overhead Dassault projection HUD was interfaced with a 
research Flight Dynamics Head-Up Guidance System® (HGS)-4000 computer. The HGS-4000 was 
modified by NASA to conduct research on certain HUD configurations. The HGS-4000 could be placed 
in a "Normal" mode, which triggered the HGS-4000 to function with its nominal commercial 
functionality, or in a “Research” mode. For this flight test, the HGS-4000 was operated in the “Research” 
mode which triggered the HGS-4000 to include some special purpose symbology, as described below. 

The HGS-4000 computer is stroke-on-raster capable using an RS-343 raster video format input. The 
HGS-4000 raster input consisted of the synthetic vision (SV) terrain and tunnel symbology while 
retaining high-quality stroke symbology for primary flight information (e.g., airspeed, altitude). The 
HGS-4000 “Primary Mode” stroke symbology set was used in the flight test, albeit with the compass rose 
symbol set removed when in “Research” mode. The raster image consisted of “layers” of imagery and 
symbology (see fig. 3). Synthetic terrain imagery formed the “Background Raster”. Guidance 
symbology and tunnel (“Pathway-in-the-Sky”) symbology were combined to create the “Foreground 
Raster”. The FOV of the ARIES HUD was measured to be 22° vertical by 28° horizontal. Note that to 
maintain conformality with the outside world, the FOV for the HUD raster image was fixed and could not 
be varied by the EP. As the minification/magnification factor was thus unity, the condition was 
colloquially known as unity FOV. 

Brightness and contrast controls were provided: a) Stroke-only brightness; b) Overall raster image 
brightness; c) Background raster (synthetic terrain imagery) contrast; d) Background raster brightness; 
and e) Foreground raster (guidance and tunnel symbology) brightness. Although somewhat complex, 
these controls gave the EP the needed flexibility to tailor the image. 

A HUD declutter button was available on the control yoke. The declutter button cycled the HUD 
symbology between four modes: 1) No declutter - All display elements present; 2) Foreground raster 
(raster guidance symbology & tunnel) removed; 3) All HUD raster removed; and, 4) All display elements 
(both stroke and raster) removed. 
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Figure 3. HUD stroke-on-raster imagery components. 


Ground-based facilities 

Flight Systems Integration Lab (FSIL). The Flight Systems Integration Lab (FSIL) at NASA 
LaRC was used to validate the experimental systems. By simulating the aircraft systems in flight 
conditions, data passing through the network devices was monitored, verifying that the test system and 
data collection was functioning properly. Evaluations in the FSIL environment were used to establish 
satisfactory performance of the HGS-4000 system. Satisfactory performance was defined as the ability to 
draw flight guidance symbology and raster imagery, provided by the SVS computers and forward-looking 
infra-red (FLIR) cameras, on the HUD combiner glass. 

Integration Flight Deck (IFD). The Integration Flight Deck (IFD) at NASA LaRC was used to 
develop and evaluate key hardware and software components, as well as provide familiarization and 
training for the flight crew prior to the EGE deployment. The IFD is a simulation facility which emulates 
the ARIES research cockpit. The IFD has the same pilot controls as ARIES. Other significant features of 
the IFD are the 6-degrees of freedom B-757 simulation model, mode control panel, and realistic 
replication of the FDRS. It also includes a representative ARIES Boeing 757 Flight Management System 
(FMS), which contains the published routes to EGE (i.e., the FMS approaches, departures, and missed 
approaches), and pilot-FMS interface controls simulation, including autoflight systems. 

Evaluation Tasks 

In general, flight-test operations involved established operational maneuvers employed by airlines 
operating at EGE. The base of operations for deployment was Colorado Springs (COS) municipal airport, 
located approximately 115 nm from EGE. COS provided minimal operational restrictions due to air 
traffic, and easy access to aircraft and personnel support facilities. All of the checkout and research flight 
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activities occurred at EGE (see fig. 4). 



Figure 4. Diagram of Eagle- Vail regional airport. 


Although EGE has received a “Special Airport” designation from the FAA, approach and landing 
procedures are not atypical of many airports constrained by terrain in all quadrants. Precision approach 
landing aides are not available due to the terrain. EGE is only equipped with an offset localizer with 
Distance Measuring Equipment (DME) (~ 1 degree offset from the runway heading) to support Localizer- 
DME Approach (LDA) procedures which includes several altitude step-downs. Visual arrivals to both 
EGE runways (07 and 25) are commenced from the East using an LDA procedure. See figures 5 and 6 
for photographs of the approach ends of the EGE runways. 
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Figure 5. Runway 25 at Eagle-Vail Regional Airport 



Figure 6. Runway 07 at Eagle-Vail Regional Airport 

To facilitate larger commercial airline operations, such as Boeing 757 aircraft, and obtain lower 
instrument approach minima, FMS-based approach and landing procedures have been developed and 
certified for EGE. These procedures also specify training and equipment standards that are well above 
that required for the FAA -published EGE instrument approach and landing procedures. 

The evaluation tasks were developed by tailoring existing FAA-approved FMS-based approach and 
departure procedures for EGE. The tailoring defined procedures and constraints which aided in 
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experimental data collection and subsequent data analysis. Two approaches and associated departure 
tasks were flown during day VMC. (See figures 7 and 8.) 

FMS Runway 25 Approach and Cottonwood-2 Departure ("FMS25"). The FMS25 task started 
on a dogleg to the final approach course, level at 13,100 ft MSL (step C in fig. 7). At the waypoint 
TALIA, approximately 16 nm from the airfield - the final approach fix - the turn to the localizer 
approach course was made and the descent into the EGE local operating area was initiated. The initial 
descent angle from TALIA was nominally 4.43 degrees. Several descent angle changes were commanded 
until approximately 1000 ft Above Field Level (AFL), where the guidance directed a 3 degree descent to 
the runway touchdown zone. A go-around was declared before 200 ft Above Ground Level (AGL) and 
the NASA SP took over control of the 757 from the EP (step 1 in fig. 7). The subsequent missed approach 
was, in fact, tailored to mirror a “worse-case” terrain-clearance departure; hence, the wording “missed 
approach” and “departure” are often interchanged throughout this document, yet they apply to the same 
task. After the go-around call, the SP performed a level-off, flew at a constant AGL over the runway, and 
reconfigured the aircraft for the climb-out. At approximately mid-field, a left turn (step J in fig. 7) was 
made to pick up the nominal FMS-departure path (the Cottonwood-2 departure) - well before the 
departure end of the runway to ensure clearance from Snow Mountain - an 1800 ft AFL peak 
approximately two nautical miles (nmi) from EGE. After the departure turn, the EP was given control of 
the aircraft and flew the Cottonwood-2 departure task. A reduced climb angle departure, loosely 
replicating the climb of a moderately loaded 757 in a single engine condition, was flown. The simulated 
single engine departure provided a worst-case operational scenario but a best-case condition for terrain 
awareness testing. The simulated single-engine Cottonwood-2 departure required a turn at Waypoint 
F219G (step L in fig. 7) to maintain terrain and obstacle clearance along Cottonwood Pass. The departure 
concluded upon reaching 10,000 ft MSL which typically occurred just prior to Waypoint F204K (step M 
in fig. 7). No departure tunnel was provided, and the EP flew conventional lateral path and speed-on- 
pitch guidance symbology for all display concepts. 

Visual Arrival to Runway 07 and KREMM Departure ("Visual 07"). The Runway 07 approach 
task started with the same FMS25 approach procedure. At approximately 5.3 nmi DME (step G1 in fig. 
8), a level off at 8100 ft MSL was commanded followed by an approximate 20 degree left turn (step HI) 
into a modified downwind leg. When about abeam the Runway 07 end, a descending right turn was 
flown for landing. For this flight test, a go-around was declared before 200 ft AGL and executed by the 
NASA SP. After the go-around call (step LI in fig. 8), the SP took over control of the 757, performed a 
level-off and reconfigured the aircraft for the climb-out. Near the departure end of the runway, the task, 
following the published KREMM departure procedure, called for an initial left turn to a 050 heading (step 
Ml in fig. 8). After the departure turn, the EP was given control of the aircraft and flew the remainder of 
the departure task (where, again, the terminology “missed approach” and “departure” may be 
interchanged, yet they relate to this same task). A reduced climb angle departure was again established to 
provide a best-case testing condition. A 050 heading was held until intercepting the 059 radial from the 
Snow VOR (Very high frequency Omni-direction Radio) beacon. The run ended upon climbing along the 
059 radial through 10,000 ft Mean Sea Level (MSL) (step Q1 in fig. 8). No departure tunnel was 
provided, and the EP flew conventional lateral path and speed-on-pitch guidance symbology for all 
display concepts. 
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Figure 7. FMS Runway 25 Approach and Cottonwood-2 Departure 



Figure 8. Visual Arrival to Runway 07 and KREMM Departure 
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Terrain Database 

The terrain databases used for this experiment were 95 nmi by 95 nmi in area, centered at the EGE 
airport. Jeppesen provided the source elevation data for the EGE databases based on United States 
Geological Survey (USGS) National Elevation Dataset (NED). The delivered elevation data was 1- 
arcsecond (30 meter) in Digital Elevation Model (DEM) format, with a Universal Transverse Mercator 
(UTM) WGS84 projection and covered a 100 nmi square geographic area centered about EGE. The 
accuracy of the source data was within 12 meters (90% of data) horizontal and 7 meters (90% of data) 
vertical. From this DEM, four real-time rendering databases were created (see fig. 9): SVS-HDD full 
color elevation-based (generic) textured, SVS-HDD full color photo-realistic textured, SVS-HUD 
monochrome green elevation-based (generic) textured, and SVS-HUD monochrome green photo-realistic 
textured. The monochrome databases were created because the HUD is monochrome. The monochrome 
databases were designed specifically for the monochrome HUD to ensure proper rendering (i.e., no holes 
in database due to lack of green color). Each EGE terrain database was built using the Terrain Experts, 
Inc (Terrex) TerraVista Pro™ software. The SV terrain databases were written to a UTM Terrex 
TerraPage™ format and rendered using CG2 VTree.™ An EGE airport model was created using 
Multigen™ Creator modeling software and placed into the SV database. 



Figure 9. Four databases with symbology overlays used for the experiment. 

To create the full color photo-realistic terrain database, multi-resolution aerial imagery (geotiff ranging 
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from 2 to 1 6 meters/pixel) was overlaid on the DEM database. The source data included three image sets 
of 1, 4. and 16 meter resolution (see table 2). The final TerraPage™ database was created from the 
source data (table 3) after a rendering trade off study. The trade off study maximized the amount of 
texturing to be rendered (most photo-realistic) while maintaining a 30 Hz update frame rate. A significant 
effort was made to color balance the various aerial images to produce a non-tiled, single brightness and 
contrast database appearance. To create the monochrome green photo-realistic database, full color aerial 
photographs were converted to a single green color using a freeware image-editing tool, ImageMagick. 

Table 2. Photo-realistic Image Sources 


Image resolution 

Provider 

Format 

Area Coverage 
(centered on EGE) 

1 -meter per pixel 

NGS 

WGS84 UTM Zone 13 

17.2 nmi (east/west) by 
6.8 nmi (north/south) 

4-meter per pixel 

ImageLinks 

geotiff WGS84 UTM Zone 13 

29.7 nmi (east/west) by 
32 nmi (north/south) 

16-meter per pixel 

ImageLinks 

geotiff WGS84 UTM Zone 13 

104.2 nmi (east/west) by 
104.2 nmi (north/south) 


Table 3. Final TerraPage Database Created from Source Data 


Image Resolution 

Coverage 

2-meter per pixel 

2. 1 nm by 0.7 nm 

4-meter per pixel 

25 nm by 25 nm 

16-meter per pixel 

95 nm by 95 nm 


To create the generic textured terrain database, a color mapping technique (i.e., “elevation shading”) 
was developed. The color scheme was based on Aeronautical Chart legends with slight modification to 
show more contrast over the elevation range in the database. The colors ranged from greens at the field 
elevation of EGE, to browns, to light tans, to off-white, with the greens representing the lower elevations 
bands, and the off-white representing the highest elevation band. Twelve bands were used, segmented 
into 250 meter ranges. To create the monochrome green generic database, shades of green were used to 
represent elevation changes. The green color intensities associated with each elevation level varied in an 
incremental fashion from the lowest to highest level. Thus, no two elevation levels had the same green 
value. Main cultural features, such as railroads, roads, lakes, and rivers, were placed in the generic 
textured terrain databases. 

Display Concepts 

The flight test was designed as a comparative study against a baseline condition. The Baseline was 
representative of the current display configuration being flown in regular airline service to EGE - the 
Boeing 757 EADI format with a TAWS-capability installed on the Navigation Display. The TAWS aural 
alerts were not implemented, as continual aural alert conditions were anticipated. In the actual test 
conditions, even the ship’s conventional Ground Proximity Warning System aural alerts had to be 
disabled. 

The Baseline display (fig. 10) was rendered on the SVS-RD. As evident in Figure 10, the Baseline 
EADI was intentionally not a direct replication of the Boeing 757 EADI but instead, was a blue-over- 
brown representation of the NASA SVS concepts. The intent was to keep the display’s symbolic 
information constant (e.g., pitch ladder) in the comparison across display concepts to avoid additional 
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variability. Also note that the ND is not directly below the EADI. The ND was offset because the control 
yoke blocked the view of the display directly below the EADI. The TAWS ND display format was 
constant across all concepts. 

Six NASA SVS concept display configurations were evaluated (SVS-HUD, SVS-HDD Size A, and 
SVS-HDD Size X, each with generic and photo-realistic terrain). A comparison of these displays (figs. 
11-15) with Figure 10 - the Baseline display configuration - shows the intuitive nature of the SVS 
display portrayal for terrain awareness. The HUD concepts utilized the Baseline concept head down, 
along with the TAWS ND. 

In addition to terrain texture differences, several important symbology differences were embedded in 
the SVS configurations. These symbology differences did not influence the terrain awareness properties 
of the display configuration evaluation but did influence the pilot’s ability to precisely monitor and 
control the aircraft. These differences include the airspeed and altitude format (e.g., “round-dials” on the 
Baseline and Size A concepts versus “tapes” on the Size X and HUD concepts), the presence (on SVS 
concepts) or absence (on Baseline) of a flight path marker (also referred to as the velocity vector), and the 
presence (on SVS concepts) or absence (on Baseline) of tunnel or “pathway-in-the-sky” information. It 
should be noted that raw data (path) indicators were provided on the glideslope and localizer deviation 
scales for all the display concepts (Baseline and SVS). These raw data indicators are referred to as 
“dogbone” indicators (see fig. 16). 

A tunnel was nominally drawn for approach guidance on the SVS-HUD and SVS-HDD (but not for 
the Baseline display condition) to increase the pilot’s awareness of the desired aircraft trajectory. The 
objective was to create path awareness yet not to obscure or occlude the terrain portrayal of the Synthetic 
Vision image. With this objective, a “minimalist” tunnel was constructed using “crow’s feet” and “goal 
posts” (see fig. 17). The crow’s feet represented the truncated corners of nominally-connected 2- 
dimensional rectangles spaced at 0.2 nm increments along the desired path. The top crow’s feet of the 
tunnel were only displayed up to 1.0 nm in front of the aircraft. The bottom crow’s feet were linearly 
decreased in brightness so, by 3.0 nm from own-ship, the brightness of the bottom crow’s feet was 
reduced to zero. The goal posts were vertical lines anchored to the ground and were spaced at 1 .0 nm 
increments along the desired path (also decreased in brightness, disappearing by 3.0 nm). 

Additional guidance information for the SVS display concepts was provided by a ghost airplane 
symbol (see fig. 18). The ghost airplane was positioned by a modified form of pursuit guidance, 
documented in Merrick (1995), to keep the aircraft trajectory tracking the tunnel. Placing the flight path 
marker on the beacon of the ghost aircraft symbol guided the EP to the desired flight path. During the 
missed approach tasks, the tunnel and ghost aircraft were removed and a single cue flight director based 
on the ship’s FMS was drawn to provide speed-on-pitch and roll steering commands (also presented on 
the Baseline EADI). 

Because of difficulties encountered with the ship systems’ FMS and flight director, approach guidance 
for the Baseline Concept varied according to Task. For the FMS 25 approach task, flight director 
guidance was provided on conventional dual cue flight director needles. For the Visual 07 approach task, 
no flight director guidance was available, and the pilots resorted to the raw lateral and vertical path 
deviation indicators (dogbones). 
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Figure 10. Baseline display, EADI with TAWS on ND. 



Figure 1 1. Size A with photo-realistic texturing. 
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Figure 12. Size A with generic texturing. 



Figure 13. Size X with photo-realistic texturing. 
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Figure 14. Size X with generic texturing. 



Figure 15. Head-Up Display with generic texturing 
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Vertical path 
(raw data) 
indicator 


Lateral path (raw data) indicator 


Figure 16. Raw data indicators for the Baseline and SVS concepts. 












Goal Post ► 
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Figure 17. Crow’s feet and goal post in the Synthetic Vision tunnel 
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Figure 18. Ghost aircraft symbol 


Experiment Design 

A full-factorial experimental matrix design was established to evaluate the six NASA SVS display 
concepts (SVS-HUD, SVS-HDD Size A, and SVS-HDD Size X, each with generic and photo-realistic 
terrain) and the Baseline display condition in each of the two approach and departure tasks. Evaluations 
were flown under simulated instrument meteorological conditions using a VRD. 

Independent Variables 

For the full-factorial experiment, the independent variables were display type (HUD, Size A, Size X, 
EADI), terrain texturing method (photo-realistic, generic), and evaluation task (Visual 07, FMS25). 

Dependent Measures for the Objective Data Analyses 

Engineering unit data was collected on all evaluations. The approach and departure paths for the 
two flying tasks (FMS25 and Visual 07) were analyzed using a flight segment analysis approach (see 
table 4 and figures 19-20). Flight segments were used since the segments contain well-defined piloting 
tasks in which specific and clear performance expectations were given to the EPs, and control of 
statistical variability could thus be anticipated. The segment definition and analyses for Segments 1 and 2 
were identical for both the FMS25 and Visual 07 tasks. The KREMM departure segments out of the 
Visual 07 task were not analyzed because the nature of the task was not conducive to a uniform flight 
conduct. The departure was not path-based (heading-based, rather than ground track) and not all of the 
runs, as actually conducted, used the exact same display configuration and flight director set-up. 
Therefore, analyses of these segments would have been more indicative of these differences than those of 
specific SVS-design parameters. 
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Table 4. Flight Segment Definitions and Associated Piloting Tasks 


Segment 

Number 

Segment 

Description 

Associated 
Flying Task 

Piloting Task 

Guidance Available 

1 

Inbound to 
Waypoint TALI A 

FMS25 & 
Visual 07 

Maintain a straight course 
(no turns) while 
remaining level at an 
altitude of 13,100 feet 
MSL 

Path deviation indicators on 
all concepts 

Baseline FMS25: Dual -cue 
flight director 

Baseline Visual 07: No flight 
director 

SVS FMS25 & Visual 07: 
Tunnel & ghost aircraft 

2 

Initial Approach 

FMS25 & 
Visual 07 

Maintain a straight course 
(no turns), and intercept 
and maintain the descent 
path 

Path deviation indicators on 
all concepts 

Baseline FMS25: Dual-cue 
flight director 

Baseline Visual 07: No flight 
director 

SVS FMS25 & Visual 07: 
Tunnel & ghost aircraft 

3 

FMS25 Final 

FMS25 

Maintain a straight course 
(no turns) and a 3.0 
degree descent path 

Path deviation indicators on 
all concepts 

Baseline: Dual-cue flight 
director 

SVS: Tunnel & ghost aircraft 

4 

Cottonwood 
Departure Initial 
Climbout 

FMS25 

Null the flight director for 
the initial straight 
climbout 

Baseline & SVS: Single-cue 
flight director 

5 

Cottonwood 
Departure Final 
Climbout 

FMS25 

Null the flight director for 
the final straight climbout 

Baseline & SVS: Single-cue 
flight director 

6 

Circle Entry Level 
off 

Visual 07 

Level off at 8,100 ft MSL 
while maintaining a 
straight course (no turns) 

Path deviation indicators on 
all concepts 

Baseline: No flight director 
SVS: Tunnel & ghost aircraft 

7 

Circle Dogleg 

Visual 07 

Maintain altitude while 
executing the dogleg left 
turn and then to maintain 
a straight and level course 
(no turns) 

Path deviation indicators on 
all concepts 

Baseline: No flight director 
SVS: Tunnel & ghost aircraft 

8 

Circling Approach 

Visual 07 

Intercept and maintain the 
3.0 degree descent path 
while executing the 
circling right turn to 
rollout on final 

Path deviation indicators on 
all concepts 

Baseline: No flight director 
SVS: Tunnel & ghost aircraft 
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Figure 19. Segmentation of FMS Runway 25 Approach and Cottonwood-2 Departure for statistical analyses. 
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descent angles 


Figure 20. Segmentation of Visual Arrival to Runway 07 for statistical analyses 

Root mean square (RMS) metrics were computed from the measures for vertical and lateral deviations 
of the B-757 from the defined path. The lateral path deviation data for the FMS25 approach runs using 
the Baseline display condition were not recorded properly. As such, this data were treated as missing 
variables in the analyses for the lateral path deviation measure. This mis-recording did not affect the 
analyses for the lateral path deviation for the Visual 07 flying task. Where available, RMS localizer error 
was also computed. When applicable, RMS flight director roll command and RMS flight director pitch 
command were also computed. 
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Analysis of the quantitative path data (RMS vertical and lateral deviation) were done by Task 
Segments and for the entire Approach. The entire Approach data included more than just the combination 
of the segment data for an approach, as some turns and some glideslope transitions were excluded from 
some of the segments by definition. The entire Approach data covered all points of the approach, from 
TALIA through to the missed approach point. 

The data were analyzed by Analysis of Variance (ANOVA) across subject, display type, texture type 
(when appropriate), and flying task. Pilot selectable FOV effects on the quantitative path data for the 
HDDs were not examined for either the individual task segments nor the entire approach task, but rather 
were accepted as a standard component of the SVS HDD display types. Within the ANOVAs, only main 
effects and second order interactions were tested. Higher order interactions were pooled into the 
experimental error term. For statistically significant factors revealed by the ANOVAs, Student-Newman- 
Keuls (SNK) tests (at a 5-percent significance level) of individual means were performed at appropriate 
stages in the analyses. 

FTE computations (which are one component of RNP calculations) were made from the recorded 
quantitative path error data for the Runway 25 and 07 approaches. These data were analyzed over the 
entire approach segment using histogram analyses. For lateral path performance, 27 bins were defined. 
Table 5 provides the bin width definitions used for the lateral path performance. For vertical path 
performance, 13 bins were defined. Table 6 provides the bin width definitions used for the vertical path 
performance. The bin values were selected to range across current-generation aircraft RNP values (> 0. 1 
nmi) with finer gradation below these values in case the advance tunnel guidance concepts provided 
measurable improvement in FTE. The number of occurrences in each bin was totaled and this total bin 
value was divided by the total number of occurrences over the entire approach to determine the 
percentage of occurrences for each bin to form the histograms. 
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Table 5. Lateral Navigation Performance Bin Definitions 


Bin 

Number 

Lateral Navigation 
Performance Range Window, x 
(nmi) 

1 

x>2.0 

2 

2.0>x>1.5 

3 

1.5>x>1.0 

4 

1.0>x>5 

5 

.5>x>.45 

6 

.45>x>.4 

7 

.4>x>.35 

8 

.35>x>.3 

9 

.3>x>.25 

10 

.25>x>.2 

11 

.2>x>.15 

12 

.15>x>.l 

13 

.l>x>.05 

14 

.05>x>-.05 

15 

-0.05>x>-0.1 

16 

-0.1>x>-0.15 

17 

-0.15>x>-0.2 

18 

-.2>x>-.25 

19 

-.25>x>-.3 

20 

-.3>x>-.35 

21 

-.35>x>-.4 

22 

-.4>x>-.45 

23 

-.45>x>-.5 

24 

-.5>x>-1.0 

25 

-1.0>x>-1.5 

26 

-1.5>x>-2.0 

27 

-2.0>x 


Table 6. Vertical Navigation Performance Bin Definitions 


Bin 

Number 

Vertical Navigation Performance 
Altitude Window, x 
(feet) 

1 

x>300 

2 

300>x>250 

3 

250>x>200 

4 

200>x>150 

5 

150>x>100 

6 

100>x>50 

7 

50>x>-50 

8 

-50>x>-100 

9 

-100>x>-150 

10 

-150>x>-200 

11 

-200>x>-250 

12 

-250>x>-300 

13 

-300>x 
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Dependent Measures for the Subjective Data Analyses 

Qualitative EP ratings and comments were collected both during the flight and in post-flight 
debriefings. In-flight pilot comments were recorded via the videotape audio recording channel. A short 
in-flight questionnaire (fig. 21) was provided aurally to the EP to elicit his comments after each run. In- 
flight comments were obtained between research maneuvers from the EP once aircraft control was 
transferred to the safety pilot. Post-flight qualitative EP ratings and other comments were obtained during 
extensive debriefings (semi-structured interviews) conducted immediately following a research flight on 
the ground at COS. 


RUN QUESTIONAIRE 


STRONGLY 

MODERATELY 

SLIGHTLY 

SLIGHTLY 

MODERATELY 

STRONGLY 

DISAGREE 

DISAGREE 

DISAGREE 

AGREE 

AGREE 

AGREE 

1 

2 

3 

4 

5 

6 


1. ) IT WAS EASY TO DETERMINE AIRCRAFT POSITION 

WITH RESPECTTO THE TERRAIN: 

COMMENTS: 

2. ) I WAS CONFIDENT IN THETERRAIN INFORMATION 

CONVEYED BY THE DISPLAY: 

COMMENTS: 

3. ) IT WAS DIFFICULT TO INTERPRET THE GUIDANCE CUES: 

COMMENTS: 

4. ) IT WAS DIFFICULT TO FOLLOW THE GUIDANCE CUES: 

COMMENTS: 

5. ) THE AMOUNT AND DENSITYOF DISPLAY 

INFORMATION WAS APPROPRIATE TO THE TASK: 

COMMENTS: 

6. ) I COULD PERFORM THIS TASK WITH EASE AND PRECISION: 

COMMENTS: 

7. ) AS I PERFORMED THETASK, THE EFFECTS OF WIND AND 

TURBULENCEWERE INCONSEQUENTIAL TO THE EVALUATION: 

COMMENTS: 


• WERE ANY EVALUATION RATINGS SIGNIFICANTLY DIFFERENT 

BETWEEN THE APPROACH AND DEPARTURETASKS? 

• OTHER COMMENTS / REM ARKS / SUGGESTIONS? 

Figure 21. In-flight run questionnaire. 

In addition to pilot ratings and comments, the Situational Awareness - Subjective Workload 
Dominance (SA-SWORD) technique (Vidulich & Hughes, 1991) was administered at the conclusion of 
the EP’s flights during their semi-structured interviews. The SA-SWORD technique uses judgment 
matrices to assess situation awareness. 
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Organization of Trials 

Approximately six evaluations per flight were planned. Each evaluation consisted of the approach and 
departure task to either Runway 25 or Runway 07. For aircraft performance considerations (fuel weight), 
Runway 07 evaluations were planned for the latter portions of each flight. Because of the desire to ensure 
collection of HUD flight test data, the HUD runs were always flown first of all the SVS display concepts. 
The experimental run matrix was developed with these constraints in place. See Appendix E for the 
planned run matrix. 

Generally, an EP’s first experimental run used the Baseline concept with no VRD installed. In fact, 
whenever the Baseline condition was evaluated, the VRD was not installed. The second experimental run 
used the HUD without the VRD installed. After the first two experimental runs were flown, the VRD 
was installed and used for the remaining display concept evaluations (except for the Baseline condition). 
The first three runs were always FMS25 runs to ensure some fuel weight reduction before attempting the 
more challenging Visual 07 runs. The SVS Size A and X concept runs, along with SVS texture type 
variations, were balanced across pilots in the usual manner to alleviate learning and fatigue effects. 

Procedure 

Upon arriving at Colorado Springs, the EP was given a briefing by a Synthetic Vision Display 
Concepts (SVDC) Principal Investigator before flying the ARIES aircraft. The briefing included the 
following elements: 

• NASA Synthetic Vision Systems Project overview 

• SVDC EGE flight test objectives 

• EGE FMS -based approaches and departures 

• SVS display concepts 

• EGE approach procedures 

• Experimental Equipment 

• Schedule 

The tests were not conducted in the blind for the EPs. All test conditions were briefed to the EP and 
the conditions had previously been flown in simulator training. Following the orientation briefing, a pre 
flight briefing was held where weather conditions and general aircraft operating procedures (e.g., 
communications, cockpit protocol, etc.) were discussed. A final general briefing was provided on the 
aircraft by the flight test director covering the sequence of maneuvers to be performed and the anticipated 
general schedule. ARIES flights originated at COS and transitioned to EGE for repetitive approach and 
departure maneuvers. ARIES transited back to COS at the end of research operations. Each flight was 
approximately four hours. 
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Results 


Objective data results are presented from ANOVAs and histogram analyses, and the subjective data 
results are presented from ANOVAs, along with any pilot comments that were considered to be 
particularly meaningful. From the ANOVAs, results for the main factors and second order interactions of 
interest are presented. Since the main factor of pilot is usually significant in these types of analyses, it is 
not specifically mentioned in the Results section unless it was found to be not significant. Within the 
ANOVAs, only main effects and second order interactions were tested. Higher order interactions were 
pooled into the experimental error term. For statistically significant factors revealed by the ANOVAs, 
SNK tests (at a 5-percent significance level) of individual means were performed at appropriate stages in 
the analyses. 

In addition, the unavailable lateral path deviation data for the Baseline condition FMS 25 runs were 
treated as missing variables for the task factor in the ANOVA analyses for this measure. As a result of 
the desire to eliminate any effects of this missing data for the Baseline Concept from the examination of 
the SVS displays, two separate ANOVAs were conducted on the quantitative path data for the individual 
task segments and the entire approach task. The first ANOVA treated display type (Baseline, Size A, 
Size X, HUD), task (FMS25, Visual 07), and pilot as the independent variables, while the second 
ANOVA treated SVS display type (Size A, Size X, HUD), texture type (generic, photo-realistic), task 
(FMS25, Visual 07), and pilot as the independent variables. Note that the second analyses are essentially 
a subset of the previous ANOVA treatments, but without the Baseline data and with different statistical 
degrees of freedom and power. Also, the second analyses enabled testing the texture type factor as well. 

Approach Path for FMS25 and Visual 07 

The analyzed approach path began at the end of Segment One (just prior to the turn at Waypoint 
TALIA), included ah turns and glideslope changes thereafter, and ended at the point where the SP took 
over control of the aircraft (before 200 ft AGL). 

Six runs (4 HUD, 1 Size A and 1 Baseline) were not included in these analyses due to known data 
contamination problems (operational restrictions, equipment problems, raster guidance symbology 
limitations, and cockpit distractions). For example, two HUD runs were excluded due to a low cloud 
ceiling of 12,500 feet MSL that prevented the pilot from flying the required altitude of 13,100 feet MSL 
during the inbound approach to Waypoint TALIA (operational restriction). Another HUD run was 
excluded due to a pilot’s inability to discern the raster guidance symbology from the raster terrain, which 
caused him to miss the initial descent at Waypoint TALIA (symbology limitation). This raster clutter 
problem (which could have been eliminated with a programmable stroke symbology capability) was 
overcome once pilots had become familiar with the condition. 

Display/Task Analyses. Separate ANOVAs were performed on the RMS lateral path deviation and 
the RMS vertical path deviation for the entire approach with display type (Baseline, Size A, Size X, 
HUD), task (FMS25, Visual 07), and pilot as the independent variables. 

Display type (F(3,61)=102.143, p<.001) was highly significant for the measure of RMS lateral path 
error during the entire approach (see fig. 22). Post hoc tests (using SNK with a=.05), showed that 
significantly worse tracking of the lateral path occurred when using the Baseline Concept (missing data 
for FMS25; raw data only for Visual 07: mean=818 ft, n=5) than when using the three SVS Concepts, 
with which the pilots had precision pathway guidance during each task: Size A (mean=61 ft, n=19), Size 
X (mean=51 ft, n=22), HUD (mean=67 ft, n=27). There were no significant differences among the SVS 
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concepts for this measure. Task, pilot, and the second order interaction of display type and task were not 
significant (p>.05) for this measure. 

Display type (F(3,65)=18.227, p<.001) was highly significant for the measure of RMS vertical path 
error during the entire approach. Post hoc tests (using SNK with a=.05) showed that the vertical path 
deviation when flying with the Baseline Concept (the pilots had differing conventional guidance 
information across the two tasks: mean=147 ft, n=10) was significantly worse than when flying with any 
of the three SVS Concepts, with which the pilots had precision pathway guidance: Size A (mean=38 ft, 
n=19), Size X (mean=40 ft, n=22), HUD (mean=32 ft, n=27). There were no significant differences 
among the SVS concepts for this measure. Task, pilot, and the interaction between display type and task 
were not significant (p>.05) for this measure. 
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Figure 22. RMS lateral and vertical path error over the entire approach path. 


SVS Display/Texture/Task Analyses. Separate ANOVAs were performed on the RMS lateral path 
deviation and the RMS vertical path deviation for the entire approach with SVS display type (Size A, Size 
X, HUD), texture type (generic, photo-realistic), task (FMS25, Visual 07), and pilot as the independent 
variables. 

Neither the main factors nor the second order interaction between SVS display type and texture type 
were significant (p>.05) for the measure of RMS lateral path error during the entire approach. SVS 
display type (F(2,56)=8.449, p=.00 1 ) and task (F(l,56)=12.884, p=.001) were significant for the measure 
of RMS vertical path error during the entire approach. Post hoc tests (using SNK with a=.05) showed 
that the vertical path deviation (fig. 22) when flying with the HUD SVS concept (mean=32 ft, n=27) was 
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significantly better than when flying with the head-down SVS Concepts: Size A (mean=38 ft, n=19) and 
Size X (mean=40 ft, n=22). The pilots had worse Packing of the vertical path during the FMS25 
approach (mean=39 feet) than with the Visual 07 approach (mean=34 feet). Texture type and the second 
order interaction between SVS display type and texture type were not significant (p>.05) for the measure 
of RMS vertical path error during the entire approach. 

FMS25 approach and departure 

Segment One: Inbound to Waypoint TALIA 

This segment was a common path for the FMS25 and Visual 07 approaches. The pilot’s task was to 
assume control of the airplane and maintain a straight course (no turns) while remaining level at an 
altitude of 13,100 feet MSL. Six runs (3 HUD, 2 Size A and 1 Baseline) were not included in these 
analyses due to known data contamination problems (operational restrictions, equipment problems, raster 
guidance symbology limitations, and cockpit distractions). 

Display/Task Analyses. Separate ANOVAs were performed on the RMS lateral path deviation and 
the RMS vertical path deviation for Segment One with display type (Baseline, Size A, Size X, HUD), task 
(FMS25, Visual 07), and pilot as the independent variables. 

Display type (F(3,14)=3.743, p=.036) was significant for the measure of RMS lateral path error (see 
fig. 23). Post hoc tests (using SNK with a=.05) showed that significantly worse tracking of the lateral 
path occurred when using the Baseline concept (missing data for FMS25; raw data only for Visual 07: 
mean=522 ft, n=5) than when using the three SVS Concepts, with which the pilots had precision pathway 
guidance: Size A (mean=116 ft, n=16), Size X (mean=lll ft, n=21), HUD (mean=157 ft, n=25). There 
were no significant differences among the SVS concepts. Task and the second order interaction of 
display type and task were not significant (p>.05) for this measure. 

Display type (see fig. 23), task, and pilot were not significant (p>.05) for the measure of RMS vertical 
path error. The second order interaction between display type and task (F(3,39)=7.904, p<.001) was 
highly significant for this measure. Examination of the interaction between display type and task (see fig. 
24) revealed poorer vertical tracking when using the Baseline Concept for the Visual 07 task (for which 
the pilots had only a raw vertical path error indicator) versus for the FMS25 task (for which the pilots had 
flight director guidance), while the task effect for the SVS Concepts (the pilots had precision pathway 
guidance during each task) was diminished and in reverse order (although these differences were not 
statistically discriminable). 
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Figure 23. RMS lateral and vertical path error over segment one. 
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Figure 24. Second order interaction of display type and task for RMS vertical path error over segment one. 
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SVS Display/Texture/Task Analyses. ANOVAs were performed on the RMS lateral path deviation 
and the RMS vertical path deviation for Segment One with SVS display type (Size A. Size X, HUD), 
texture type (generic, photo-realistic), task (FMS25, Visual 07), and pilot as the independent variables. 
Neither the main factors nor the interaction between SVS display type and texture type were significant 
(p>.05) for the measures of RMS lateral path error and RMS vertical path error during this segment. 

Segment Two: Initial Approach 

This segment was a common path for the FMS25 and Visual 07 approaches. The pilot’s task was to 
maintain a straight course (no turns), and intercept and maintain the descent path. Five runs (3 HUD, 1 
Size A and 1 Baseline) were not included in these analyses due to known data contamination problems 
(operational restrictions, equipment problems, raster guidance symbology limitations, and cockpit 
distractions). 

Display/Task Analyses. ANOVAs were performed on the RMS lateral path deviation, the RMS 
localizer error, and the RMS vertical path deviation for Segment Two, with display type (Baseline, Size 
A, Size X, HUD), task (FMS25, Visual 07), and pilot as the independent variables. 

For Segment Two, display type (F(3,14)=15.628, p<.001) was highly significant for the measure of 
RMS lateral path error (see fig. 25). Post hoc tests (using SNK with a=.05), showed that lateral path 
tracking performance using the Baseline concept (missing data for FMS25; raw data only for Visual 07: 
mean=867 ft, n=5) was significantly worse than when using the three SVS Concepts, with which the 
pilots had precision pathway guidance: Size A (mean=42 ft, n=19), Size X (mean=29 ft, n=22), HUD 
(mean=52 ft, n=27). There were no significant differences among the SVS concepts for this measure. 
Task, pilot, and the second order interaction between display type and task were not significant (p>.05) 
for this measure. 

Display type (F(3,15)=4.964, p=.014), task (F(l,5)=9.0, p=.03), and the interaction between display 
type and task (F(3,45)=15.406, p<.001) were significant for the measure of RMS localizer error during 
Segment Two. Post hoc tests (using SNK with a=.05) showed that the localizer tracking when using the 
Baseline concept (the pilots had differing conventional guidance information across the two tasks: 
mean=.349 dots, n=10) was significantly worse than when using the three SVS Concepts, with which the 
pilots had precision pathway guidance: Size A (mean=.061 dots, n=19), Size X (mean=.061 dots, n=22), 
HUD (mean=.064 dots, n=27). There were no significant differences among the SVS concepts. Pilots 
accrued less localizer error during the Visual 07 approach (mean=.097 dots) than when flying the FMS25 
approach (mean=. 101 dots). Although this result is statistically significant, it was not considered 
operationally meaningful as the difference between the two means was only .004 dots. Examination of 
the interaction between display type and task (see fig. 26) revealed that task differences for the Baseline 
were large compared to the differences for the three SVS Concepts. Larger localizer error with the 
Baseline display was obtained during the Visual 07 task (for which the pilots had only a raw localizer 
error indicator) than during the FMS25 task (for which the pilots had flight director guidance). Note that 
because no lateral path error data was available for the Baseline Concept for the FMS25 task (treated as 
missing data), a comparable statistically significant interaction was not possible for the previous analysis 
of the RMS lateral path error measure (lateral path error is a linear measure and as such is a more 
sensitive measure of path performance than the angular localizer error measure). The remaining main 
factor, pilot, was not significant (p>.05) for this measure. 

Display type (F(3,15)=8.593, p=.014) was highly significant for the measure of RMS vertical path 
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error during Segment Two (see fig. 25). Post hoc tests (using SNK with oc=.05) showed that the vertical 
path tracking performance using the Baseline concept (the pilots had differing conventional guidance 
information across the two tasks: mean=185 ft, n=10) was significantly worse than the tracking 
performance when using the three SVS Concepts, with which the pilots had precision pathway guidance: 
Size A (mean=39 ft, n=19), Size X (mean=42 ft, n=22), HUD (mean=39 ft, n=27). There were no 
significant differences among the SVS concepts. Task, pilot, and the second order interaction between 
display type and task were not significant (p>.05) for this measure. 
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Figure 25. RMS lateral and vertical path error over segment two (initial approach). 
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Segment Two RMS Localizer Error 



Figure 26. RMS localizer error plot of display type and task interaction over segment two. 


SVS Display/Texture/Task Analyses. ANOVAs were performed on the RMS lateral path deviation, 
the RMS localizer deviation, and the RMS vertical path deviation for Segment Two with SVS display 
type (Size A, Size X, HUD), texture type (generic, photo-realistic), task (FMS25, Visual 07), and pilot as 
the independent variables. 

Neither the main factors nor the second order interaction between display type and texture type were 
significant (p>.05) for the measure of RMS lateral path error or for the measure of RMS vertical path 
error. For the measure of RMS localizer, SVS display type, texture type, and the interaction between 
SVS display type and texture type were not significant (p>.05), but task was significant (F( 1 ,5)=3.0, 
p=.0 1 I j. Pilots accrued less localizer error during the Visual 07 approach (mean=.054 dots) than when 
flying the FMS25 approach (mcan=.073 dots). Although this result is statistically significant, it was not 
considered operationally meaningful. The lateral flight plans for the Visual 07 and FMS25 over Segment 
Two differed by 0.02 degrees, which, at EGE’s scaling of 1.5deg/dot, resulted in a 0.013 dots difference. 
Because of that slight path definition difference, the RMS localizer is biased by 0.013 dots between the 
two tasks. The statistical differences noted above (0.073-0.054=0.019 dots) are thus attributed to this path 
mismatch, rather than to display effects. It should be noted that the path mismatch influences on localizer 
error just discussed had negligible effect on the results previously presented on the Baseline condition 
task findings for localizer error, as the task effect was much larger and in the opposite direction there (see 
fig. 26: 0.25-0.45=-0.20 dots). 

Segment Three: FMS25 Final Approach 

This critical segment was the straight final for the FMS 25 approach. It typically ended in a 
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takeoff/go-around (TOGA) just before the left turn away from the offset localizer course to align with the 
runway at 200 ft AFL. The pilot’s task was to maintain a straight course (no turns) and a 3.0 degree 
descent path. Two runs (1 HUD and 1 Baseline) were not included in these analyses due to known data 
contamination problems caused by equipment problems. 

Display Analyses. ANOVAs were performed on the RMS vertical path deviation and the RMS 
localizer error for Segment Three with display type (Baseline, Size A, Size X, HUD) and pilot as the 
independent variables. An ANOVA was not performed for the lateral path deviation with display type as 
an independent variable because there was no Baseline FMS25 data available for this measure. However, 
an ANOVA was performed on this measure with SVS display type as an independent variable and that 
analysis is presented in the next section. 

Display type and pilot were not significant (p>.05) for the measure of RMS vertical path deviation. 
Display type (F(3, 1 1)=8. 1 82, p=.004) was highly significant for the measure of RMS localizer error (see 
fig. 27), an angular measure, that while less sensitive than the linear lateral path deviation measure, 
provides related results. Post hoc tests (using SNK with a=.05) showed that the pilot accrued 
significantly more localizer error with the Baseline concept (using a conventional flight director, 
mean=.320 dots, n=5) than when using the three SVS Concepts (using precision pathway guidance): Size 
A (mean=.068 dots, n=8), Size X (mean=.070 dots, n=10), HUD (mean=.065 dots, n=l 1). There were no 
significant differences among the SVS concepts. Pilot was not significant (p>.05) for the measure of 
RMS localizer error. 
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Figure 27. RMS localizer error over segment three (FMS25 short final) 
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SVS Display/Texture Analyses. ANOVAs were performed on the RMS lateral path deviation and 
the RMS vertical path deviation for Segment Three with SVS display type (Size A, Size X, HUD), texture 
type (generic, photo-realistic), and pilot as the independent variables. 

SVS display type (F(2,7)=l 1 .959, p=.006) was highly significant for the measure of RMS lateral path 
error (see fig. 28). Post hoc tests (using SNK with a=.05), showed that the pilot had worse tracking of the 
lateral path with the HUD (mean=36 ft, n=l 1) concept than when using the other two SVS Concepts: Size 
A (mean=l 1 ft, n=8) and Size X (mean=12 ft, n=10). One could postulate that the poorer lateral tracking 
when using the HUD could be attributed to learning effects with the use of the vision restriction device 
(VRD). In general, the pilots’ first HUD run was always without the VRD (all baseline runs were without 
the VRD and, for safety considerations, usually preceded the first HUD run to a particular runway end). 
Once the VRD was installed, the mean for the RMS lateral deviation when using the HUD concept (each 
pilot’s second HUD run; mean=55 feet, n=4) increased as compared to using the HUD without the VRD 
installed (each pilot’s first HUD run; mean=31 feet, n=4). Again, one could postulate that there was a 
learning effect as the pilots’ tracking became better as they got used to flying with the VRD installed 
(each pilot’s third HUD run; mean=17 feet, n=3). However, such an effect was not exhibited for the 
vertical error tracking when using the HUD with and without the VRD. The VRD was in place for all the 
Size A and X concept runs, and the Size A and X concept runs, along with texture type runs, were 
balanced across pilots. Conversely, turbulence effects were greater for the later runs of a sortie, such that 
one could argue the Size A and X concept run conditions might have been more difficult than the HUD 
runs. Consequently, no concrete explanation can be offered for the exceptionally good RMS lateral path 
tracking performance obtained over the 3.83 Nmi Segment Three when using the Size A (mean=ll ft, 
n=8) and Size X (mean=12 ft, n=10) concepts. Pilot, texture type and the interaction between SVS 
display type and texture type were not significant (p>.05) for the RMS lateral path error measure. 
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Figure 28. RMS lateral path error over segment three (FMS25 final). 
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SVS display type (see fig. 28), texturing type, pilot, and the interaction between texture type and SVS 
display type were not significant (p>.05) for the measure of RMS vertical path error during the straight 
final approach to Runway 25. 

Segment Four: Cottonwood Departure Initial Climbout 

This segment was the initial straight climbout for the Cottonwood departure of the FMS25 flying task. 
The pilot’s task was to null the flight director. The flight director commands were based on LNAV and 
Speed on Pitch. 

Display Analyses. ANOVAs were performed on the RMS flight director roll command and the RMS 
flight director pitch command with display type (Baseline, Size A, Size X, HUD) and pilot as the 
independent variables. Display type was not significant (p>.05) for either measure. 

SVS Display/Texture Analyses. ANOVAs were performed on the RMS flight director roll command 
and the RMS flight director pitch command with SVS display type (Size A, Size X, HUD), texture type 
(generic, photo-realistic), and pilot as the independent variables. SVS display type, texture type, pilot, 
and the interaction between SVS display type and texture type were not significant (p>.05) for either 
measure. 

Segment Five: Cottonwood Departure Final Climbout 

This segment was the final straight climbout for the Cottonwood departure of the FMS25 flying task. 
The pilot’s task was to null the flight director. The flight director commands were based on LNAV and 
Speed on Pitch. 

Display Analyses. ANOVAs were performed on the RMS flight director roll command and the RMS 
flight director pitch command with display type (Baseline, Size A, Size X, HUD) and pilot as the 
independent variables. Display type was not significant (p>.05) for either measure and pilot was not 
significant (p>.05) for the flight director pitch command measure. 

SVS Display/Texture Analyses. ANOVAs were performed on the RMS flight director roll command 
and the RMS flight director pitch command with SVS display type (Size A, Size X, HUD), texture type 
(generic, photo-realistic), and pilot as the independent variables. SVS display type, texture type, pilot, 
and the interaction between SVS display type and texture type were not significant (p>.05) for either 
measure. 

Visual 07 approach 

Segments one and two of this approach were covered in the section entitled FMS25 Approach and 
Departure, as these two segments were common to both flying tasks. 

Segment Six: Circle Entry Level off 

This segment was the level off before the circling approach to runway 07. The pilot’s task was to 
level off at 8,100 ft MSL while maintaining a straight course (no turns). 

Display Analyses. ANOVAs were performed on the RMS lateral path deviation and the RMS vertical 
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path deviation for Segment Six with display type (Baseline, Size A, Size X, HUD) and pilot as the 
independent variables. 

Display type (F(3,14)=5.350, p=. 012) was highly significant for the measure of RMS lateral path error 
(see fig. 29). Post hoc tests (using SNK with a=.05) showed that the pilots had worse tracking of the 
lateral path with the Baseline concept (for which the pilots had only a raw lateral path error indicator: 
mean=203 ft, n=5) than when using the three SVS concepts, with which the pilots had precision pathway 
guidance: HUD (mean=46 ft, n=19), Size X (mean=18 ft, n=12) and Size A (mean=28 feet, n=12) 
concept. There were no significant differences among the SVS concepts. The pilot factor was not 
significant (p>.05) for the RMS lateral path error measure. 

Display type (F(3,14)=12.953, p<.001) was highly significant for the measure of RMS vertical path 
error during Segment Six (see fig. 29). Post hoc tests (using SNK with a=.05) showed that the pilot had 
worse tracking of the vertical path with the Baseline concept (for which the pilots had only a raw vertical 
path error indicator: mean= 1 22 ft, n=5 ) than when using the three SVS Concepts, with which the pilots 
had precision pathway guidance: Size A (mean=59 ft, n=12), Size X (mean=50 ft, n=13), HUD (mean=38 
ft, n=19). Also, tracking with the HUD concept was significantly better than with the Size A concept but 
performance with the HUD could not be discriminated from that of the Size X concept. The Size X 
concept performance could also not be discriminated from that of the Size A concept. The pilot factor 
was not significant (p>.05) for the RMS vertical path error measure. 
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Figure 29. RMS lateral and vertical path error over segment six (circle entry level off). 

SVS Display Type/Texture Analyses. ANOVAs were performed on the RMS lateral path deviation 
and the RMS vertical path deviation for Segment Six with SVS display type (Size A, Size X, HUD), 
texture type (generic, photo-realistic), and pilot as the independent variables. SVS display type 
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(F(2,10)=5.425, p=.025) was significant for the measure of RMS lateral path error. However, post hoc 
tests (using SNK with a=.05) could not discriminate between the three SVS concepts for this measure. 
The SNK uses a different statistical model than the ANOVA and in this case had less power to 
discriminate between the means. (See Display Analyses section above and fig. 29 for the means.) 
Texture type, pilot and the interaction between SVS display type and texture type were not significant 
(p>.05) for the RMS lateral path deviation measure. 

SVS display type, texture type, the interaction between SVS display type and texture type, and pilot 
were not significant (p>.05 ) for the measure of RMS vertical path error. 

Segment Seven: Circle Dogleg 

This segment was the level dogleg left turn prior to the right circling approach to runway 07. The 
pilot’s task was to maintain altitude while executing the turn and then to maintain a straight and level 
course (no turns). 

Display Analyses. ANOVAs were performed on the RMS lateral path deviation and the RMS vertical 
path deviation for Segment Seven with display type (Baseline, Size A, Size X, HUD) and pilot as the 
independent variables. 

Display type (F(3, 14)= 19. 133, p<.001) was highly significant for the measure of RMS lateral path 
error (see fig. 30). Post hoc tests (using SNK with oc=.05) showed that lateral path control using the 
Baseline concept (for which the pilots had only a raw lateral path error indicator: mean=522 ft, n=5) 
concept was significantly worse than when using the three SVS Concepts, with which the pilots had 
precision pathway guidance: Size A (mean=35 ft, n=12), Size X (mean=39 ft, n=12), HUD (mean=50 ft, 
n=19). There were no significant differences among the SVS concepts for the RMS lateral path deviation 
measure. The pilot factor was not significant (p>.05) for the RMS lateral path error measure. 

Display type (F(3,14)=2 1.063, p<.001) was also highly significant for the measure of RMS vertical 
path error during Segment Seven (see fig. 30). Post hoc tests (using SNK with a=.05) showed that the 
pilots had worse tracking of the vertical path with the Baseline concept (for which the pilots had only a 
raw vertical path error indicator: mean=86 ft, n=5) than with any of the three SVS Concepts, with which 
the pilots had precision pathway guidance: Size A (mean=19 ft, n=12), Size X (mean=18 ft, n=12), HUD 
(mean=ll ft, n=19). There were no significant differences among the SVS concepts. The pilot factor 
was not significant (p>.05) for the RMS vertical path error measure. 
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Figure 30. RMS lateral and vertical path error over segment seven (circle dogleg). 
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SVS Display/Texture Analyses. ANOVAs were performed on the RMS lateral path deviation and 
the RMS vertical path deviation for Segment Seven with SVS display type (Size A, Size X, HUD), 
texture type (generic, photo-realistic), and pilot as the independent variables. SVS display type, texture 
type, the interaction between these 2 main factors, and pilot were not significant (p>.05) for either 
measure. 

Segment Eight: Circling Approach 

This segment was the circling, descending approach to runway 07 with a rollout on short final at about 
350 ft AFL. The pilot’s task was to intercept and maintain the 3.0 degree descent path while executing 
the circling right turn to rollout on final. This critical segment typically ended in a TOGA just before 200 
ft AFL. 

Display Analyses. ANOVAs were performed on the RMS lateral path deviation and the vertical path 
deviation for Segment Eight with display type (Baseline, Size A, Size X, HUD) and pilot as the 
independent variables. 

Display type (F(3,14)=17.627, p<.001) was highly significant for the measure of RMS lateral path 
error (see fig. 31). Post hoc tests (using SNK with a=.05), showed that the pilot had worse tracking of the 
lateral path with the Baseline concept (for which the pilots had only a raw lateral path error indicator: 
mean=685 ft, n=5) than when using the three SVS Concepts, with which the pilots had precision pathway 
guidance: Size A (mean=83 ft, n=12), Size X (mean=100 ft, n=12), and HUD (mean=55 ft, n=19). Also, 
tracking with the HUD concept was significantly better than with the Size X concept but performance 
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with the HUD could not be discriminated from that of the Size A concept. The Size A concept 
performance could also not be discriminated from that of the Size X concept. The pilot factor was not 
significant (p>.05) for the RMS lateral path deviation measure. 

Display type (F(3,14)= 5.610, p=.010) was also highly significant for the measure of RMS vertical 
path error during Segment Eight (see fig. 31). Post hoc tests (using SNK with a=.05) showed that 
vertical path control was worse when using the Baseline concept (for which the pilots had only a raw 
vertical path error indicator: mean=78 ft, n=5) than when using the three SVS Concepts, with which the 
pilots had precision pathway guidance: Size A (mean=40 ft, n=12), Size X (mean=38 ft, n=12), and HUD 
(mean=20 ft, n=19). Also, tracking with the HUD concept was significantly better than with the Size A 
concept but performance with the HUD could not be discriminated from that of the Size X concept. The 
Size X concept performance could also not be discriminated from that of the Size A concept. The pilot 
factor was not significant (p>.05) for the RMS vertical path deviation measure. 
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Figure 31. RMS lateral and vertical path error over segment eight (circling approach). 


SVS Display Type/Texture Analyses. ANOVAs were performed on the RMS lateral path deviation 
and the RMS vertical path deviation for Segment Eight with SVS display type (Size A, Size X, HUD), 
texture type (generic, photo-realistic), and pilot as the independent variables. 

The SVS-only display type analysis for the measure of RMS lateral path error is essentially a subset of 
the previous ANOVA treatment, but without the Baseline data and with different statistical degrees of 
freedom and power. Post hoc tests (using SNK with a=.05), showed that tracking with the HUD concept 
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(mean=55 ft, n=19) was significantly better than with the Size X concept (mean= 1 00 ft, n=l 2), but 
performance with the HUD could not be discriminated from that of the Size A concept (mean=83 ft, 
n=12). The Size A concept performance could also not be discriminated from that of the Size X concept. 
The interaction between SVS display type and texture type (F(2,17)=3.854, p=042) was significant for 
the measure of RMS lateral path error. Examination of the interaction between SVS display type and 
texture (see fig. 32) revealed poorer lateral tracking using the Size X Photo-realistic concept than when 
using the Size X Generic Concept, while the texture effect for the Size A and HUD SVS Concepts was 
diminished and in reverse order (although not statistically discriminable). Texture type and pilot were not 
significant (p>.05) for the measure of RMS lateral path error. 

SVS display type (F(2,10)=10.139, p=.004) was highly significant for the measure of RMS vertical 
path error. Post hoc tests (using SNK with oc=.05) showed that the pilots had better tracking of the 
vertical path with the HUD concept (mean=20 ft, n=19) than when using the Head-down SVS Concepts: 
Size A (mean=40 ft, n= 1 2) and Size X (mean=38 ft, n=12). There were no differences among the head- 
down concepts for this measure. Texture type and the interaction between SVS display type and texture 
type were not significant (p>.05) for the measure of RMS vertical path error. 
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Figure 32. RMS lateral error plot of SVS display type and texture interaction. 

Lateral Navigation Bin Analyses 

The system performance requirements for RNAV is that each aircraft operating in RNP airspace shall 
have total (lateral) system error components in the cross-track and along-track directions that are less than 
the RNP value 95% of the flying time (see Appendix D). For the reasons cited previously in the 
Approach Path for FMS25 and Visual 07 section, six runs were excluded from the Lateral Navigation 
analyses. Figures 33-36 show the horizontal FTE distribution for the display concepts using the bin width 
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definitions provided in table 5. The path steering error component of the RNP calculation includes both 
FTE and display error. For this analysis, it was assumed that display error was negligible, so FTE was the 
only component of path steering error. It was also assumed that the other two components (path 
definition error and position estimation error) of the RNP calculation would be equivalent across the 
display concepts evaluated. 

With these assumptions, the SVS concepts yielded a horizontal FTE navigational accuracy of 0.05 nmi 
at least 95% of the time; while the Baseline concept was only able to yield a horizontal FTE navigational 
accuracy of 0.25 nmi at least 95% of the time. As such, based on the FTE distributions shown in figures 
33-36, the SVS concepts with precision pathway information (Size A, Size X, HUD) would enable 
horizontal RNP-type operations that were five times smaller than those that would be allowed with the 
Baseline EADI concept. 
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Figure 33. Lateral FTE distribution for the Baseline EADI concept 
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Figure 34. Lateral FTE distribution for the Size A SYS concept 
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Figure 35. Lateral FTE distribution for the Size X SYS concept 


48 


</> 

<D 

O 

c 

0 


Lateral FTE - Approach 
HUD 


100 

90 

80 

70 


O) 

CJ) 


60 


3 
O 

o 50 


40 


30 


0 
n 

1 20 

z 

10 

0 


OOOOOOOOOOOOLO 

qqqqqqqqqqqqcy 

ooooooooooooo 


CO CD O O 
-i 1 O O 


o o o 
o o o 


o o o 
o o o 


o o o 
o o o 


ooooooooooooo 


LO 

O 

LO 

LO 


LO 

CO 

LO 

CM 

LO 


LO 

LO 

i— 

LO 

CM 

LO 

CO 

LO 


LO 

LO 

T— 

LO 

CM 

O 

T— 

T— 

A 

''t 

A 

CO 

A 

CM 

A 


A 

O 

O 

o 

T 

O 

CM 


CO 

i' 

'=t 

i‘ 

' 

T— 

A 

CM 

A 

X 

A 

CM 

A 

X 

A 

LO 

X 

A 

O 

A 

X 

A 

LO 

X 

A 

LO 

'St 

A 

X 

A 

-sf 

X 

A 

LO 

CO 

A 

X 

A 

CO 

X 

A 

LO 

CM 

A 

X 

A 

CM 

X 

A 

LO 

A 

X 

A 

A 

X 

A 

LO 

A 

X 

A 

LO 

o 

A 

X 

A 

A 

X 

A 

LO 

A 

X 

A 

CM 

X 

A 

LO 

CM 

A 

X 

A 

CO 

X 

A 

LO 

CO 

A 

X 

A 

X 

A 

LO 

X 

A 

LO 

A 

X 

A 

X 

A 

LO 

V 

X 


T— 











O 

O 


i— 

i' 

i' 

i' 

i' 

i' 

i' 


1 

1 



o o o 


Lateral Bins, nmi 


Figure 36. Lateral FTE distribution for the F1UD SVS concept 

Vertical Navigation Bin Analysis 

The system performance requirements for VNAV are that for at least 99.7% of the time the 
navigational performance in the vertical plane, or the total vertical system error, is less than a specified 
altitude deviation measure (see Appendix D). For the reasons cited previously in the Approach Path for 
FMS25 and Visual 07 section, six runs were excluded from the Vertical Navigation analyses. Figures 37- 
40 show the vertical FTE distribution for the display concepts using the bin width definitions provided in 
table 6. The vertical path steering error component of the VNAV performance calculation includes both 
FTE and display error. For this analysis, it was assumed that display error was negligible so FTE was the 
only component of vertical path steering error. It was also assumed that the other three components 
(altimetry system error, vertical path definition error, and horizontal coupling error) of the VNAV 
performance calculation would be equivalent across the display concepts evaluated. In addition, it was 
assumed that the pilot was flying a specified vertical profile so that the required vertical navigation 
performance accuracy was 300 feet (see table 1). 

The HDD SVS concepts (Size A, Size X) yielded a vertical FTE navigational accuracy of 150 feet at 
least 99.7% of the time and the HUD SVS concept yielded a vertical FTE navigational accuracy of 100 
feet at least 99.7% of the time. The Baseline concept was unable to yield a vertical FTE navigational 
accuracy of 300 feet for at least 99.7% of the time. As such, based on the FTE distributions shown in 
figures 37-40, the SVS concepts with precision pathway information (Size A, Size X, HUD) would 
enable RNP-type operations along a specified vertical profile of 300 feet and the Baseline EADI concept 
would not. Thus, the SVS concepts enhance flight operations by enabling the specification of a flight 
path vertically for a given lateral flight path. 
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Figure 37. Vertical FTE distribution for the Baseline EADI concept 
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Figure 38. Vertical FTE distribution for the Size A SYS concept 
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Figure 39. Vertical FTE distribution for the Size X SYS concept 
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Figure 40. Vertical FTE distribution for the HUD SYS concept 
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Subjective Data Analyses 


Post-Run Ratings 

Seven post-run questions (see fig. 21) were asked of each evaluation pilot to help assess specific 
subjects of interest to researchers while flying the approaches with the display concepts. Only those 
questions of particular interest with regard to terrain awareness and pilot workload are discussed here 
(questions #1,2, and 6). An ANOVA was performed on the mean rating (1 = “strongly disagree”; 6 = 
“strongly agree”) for each of those post-run questions with display type (seven levels: Baseline; and Size 
A, Size X, and HUD, each with generic and photo-realistic terrain) and pilot as the independent variables. 
The pilot factor is not specifically mentioned unless it was found not to be significant. 

Terrain Awareness. Post-run question # 1, “It was easy to determine aircraft position with respect to 
the terrain”, was asked of each EP to help assess his terrain awareness when flying the different display 
concepts. Display type (F(6,73)=2.82, p=.016) was highly significant for the measure of the mean terrain 
awareness rating (see fig. 41). Post hoc tests (using SNK with a=.05) showed that the pilots felt that their 
terrain awareness while using the Size A Photo-realistic, Size X Generic, and Size X Photo-realistic 
concepts was significantly better than when using the Baseline E ADI/TAWS ND concept. No other 
differences could be discriminated. 

Post-run question # 2, “I was confident in the terrain information conveyed by the display”, was asked 
of each EP to help assess his confidence in the terrain information when flying the different display 
concepts. Display type (F(6,73)=3.30, p=.006) was highly significant for the measure of the mean terrain 
confidence rating (see fig. 41). Post hoc tests (using SNK with a=.05) showed that the pilots felt 
significantly more confident in the terrain information provided by the Size X Generic and Size X Photo- 
realistic concepts than that of the Baseline concept. No other differences could be discriminated. 

Workload. Post-run question # 6, “I could perform this task with ease and precision”, was asked of 
each EP to help assess his workload when flying the different display concepts. Display type 
(F(6,73)=5.594, p<.000) was highly significant for the measure of the mean workload rating (see fig. 41). 
Post hoc tests (using SNK with a=.05), showed that use of the Baseline concept (mean=4. 1) imposed 
significantly more workload on the pilots than the use of any of the SVS concepts: Size A Generic 
(mean=5.2), Size A Photo-realistic (mean=5.7), Size X Generic (mean=5.7), Size X Photo-realistic 
(mean=5.4), HUD Generic (mean=5.5), and HUD Photo-realistic (mean=5.7). There were no significant 
differences among the SVS concepts for this measure. 
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In-Flight Questionnaire Ratings 
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Figure 4 1 . Post-run mean pilot ratings 


Display Questionnaire Ratings 

Appendix F documents the mean and standard deviations for each of the 70 post-flight questionnaire 
ratings on specific subjects of interest to researchers. Those of particular interest with regard to pilot SA 
(questions 11-14, 43-46, and 67-70) and pilot workload (questions 8-9, 39-42, and 63-66) using SVS 
display concepts are presented here. Due to an omission in the questionnaire, the pilots were not asked to 
rate the workload for the HUD Photo-realistic concept. Mean pilot ratings of situation awareness and 
workload (see figures 42 and 43, respectively) for both the head-up and head-down SVS display concepts 
tested indicated that all of these concepts provided enhanced situation awareness while imposing low 
workload for the pilot. Statistical analyses of these results were not conducted. 
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Figure 42. Mean pilot situation awareness ratings versus display type 
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Figure 43. Mean pilot workload ratings versus display type 


SA-SWORD Ratings 

Pilots were asked to complete a paired-comparison SA-SWORD (Vidulich & Hughes, 1991). The 
SA-SWORD for this experiment was designed to allow a statistical analysis of the pilot’s subjective 
assessment of the situation awareness for each of the display configurations (Baseline, Size A Generic, 
Size A Photo-realistic, Size X Generic, Size X Photo-realistic, HUD Generic and HUD Photo-realistic). 
For this exercise, SA was defined as: The pilot’s awareness and understanding of all factors that will 
contribute to the safe flying of their aircraft under normal and non-normal conditions. 

The responses were averaged and the overall rank order was: Size X Photo-realistic, Size X Generic, 
HUD Photo-realistic, Size A Photo-realistic, Size A Generic, HUD Generic, and Baseline. An ANOVA 
was performed on the mean rankings with display type and pilot as the independent variables. Display 
type (F(6,18)=6.968, p<.001) was highly significant for this measure. Post hoc tests (using SNK with 
a=.05) showed that the Size X Photo-realistic had significantly higher SA-SWORD ratings than all other 
SVS display concepts except for Size X Generic. Three distinct, overlapping subsets (see fig. 44) were 
formed: 1) Size X Photo-realistic & Size X Generic; 2) Size X Generic & HUD Photo-realistic; and 3) 
HUD Photo-realistic, Size A Photo-realistic, Size A Generic, HUD Generic, & Baseline. 
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Figure 44. Comparative situation awareness among display concepts. 


Pilot Comments 


Video and audio recordings of each pilot were taken while they flew the two approaches and 
departures. In addition, each pilot’s post-flight debriefing was recorded on audiotapes. 

There were numerous comments on elements of the tunnel symbology but they will not be discussed 
here as the emphasis of this flight test was on the presentation of terrain information. In summarizing 
these pilot comments, the following themes emerged. 


1 . All but one pilot felt that the NASA S VS HUD concepts were viable for presenting terrain 
information to the pilot. One pilot commented that it gave a significant improvement in 
situation awareness over the Baseline EADI concept because of the terrain. Two of the pilots 
stated they would like the ability to toggle the terrain information on and off. The one 
differing pilot felt that a wireframe terrain HUD concept would be an improvement over the 
NASA HUD concepts because there are occasions when you want to see through the HUD 
raster image. 

2. For the HDDs, all pilots overwhelming preferred the larger size display (Size X). In fact, 
three of the pilots commented that the Size X photo-realistic textured SVS concept provided 
the most enhanced situation awareness of the concepts evaluated. The pilots also stated that 
size (larger is better) is more important than the texturing method for the SVS concepts 
evaluated. 

3. For the HDDs, each pilot agreed that workload, situation awareness, and pilot interpretation of 
performance parameters (e.g., airspeed, vertical speed information, etc.) was not affected by 
the type of texturing, photo-realistic or generic, employed on the SVS concept. However, four 
of the pilots said that they would choose photo-realistic over generic texturing for each of the 
HDDs evaluated. 

4. Two of the pilots suggested that using both generic and photo-realistic texturing within one 
database would have merit as each has its advantages. Photo-realistic texturing gives a pilot 
an excellent view of cultural features (hills, roads, vegetation) and would be useful near the 
airport during approaches and departures. Generic texturing shows a pilot the angular 
relationship to the view of the terrain, thus enhancing the elevation cues, and would be useful 
when flying enroute segments. 
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5. Two of the pilots suggested placing waypoints with altitude information on the SVS primary 
flight displays. Then they could crosscheck with the Navigation Display when passing the 
waypoints. 

Of particular interest was a comment made by one pilot after completing a Visual 07 approach with 
the Size A Photo -realistic concept in simulated IMC: 

“I often commented to people over the years that I never ever flew a circling approach in the 141 that 
1 was ever comfortable with, particularly at night. It always demanded a lot of attention. This was the 
first time I ever had an occasion of circling an approach with the kind of information 1 would love to have 
in a circling approach. Keeping me safe, 1 could see the terrain, taking me where 1 want to go, getting me 
all types of information in terms to where I am relative to the end of the runway. 1 mean it’s the best of 
all possible worlds in terms of safety." 
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Discussion 


The NASA AvSP SVS Project has been focused on developing new technologies that will mitigate, or 
even eliminate, visibility-induced accidents through the cost-effective use of synthetic vision displays. 
Additionally, Synthetic Vision has been forecast to reduce FTE, increase RNP capabilities, and provide 
other significant economic benefits. Therefore, SVS has many rewards associated with its development 
and implementation. To achieve such considerable potential, research is needed to deliver products which 
fulfill these technology objectives. Moreover, the introduction of such an innovative display concept may 
have unforeseen risks which requires data to safeguard the position that SVS supports “human-centered” 
design. Past research has consistently demonstrated that SVS is a substantial improvement over current 
cockpit instrumentation, and no human factors shortcomings have been uncovered. However, because 
one major objective of SVS is to enhance safety and operational capabilities in terrain environments, 
testing at a terrain-challenged location - in this case, Eagle-Vail, Colorado - was warranted. 

The NASA flight test had several objectives to demonstrate the potential of SVS technology. The 
objectives of the research were: 

a) Confirm the potential of the NASA SVS HUD concept as a retrofit solution for display of SVS 
concepts in non-glass cockpits. Determine the potential in both day VMC and in day, low- 
visibility operational environments. 

b) Confirm results from piloted simulation experiments and the SVS-DFW flight test for 
operational utility and acceptability of various-sized (Size A, X) head-down synthetic vision 
displays. 

c) Compare the operational utility and acceptability of photo-realistic textured with generic 
textured terrain databases within the NASA SVS concepts (HUD; head-down Size A, X). 

d) Assess pilot path control performance (flight technical error) during manually flown landing 
approach and go-around maneuvers in a terrain-challenged operational environment, with and 
without SVS display concepts. 

e) Determine required navigation performance capabilities with SVS display concepts for area 
navigation. 

f) Confirm the situation awareness and workload benefits of SVS display concepts. 

g) Provide demonstration of the economic potential of SVS for approaches that have significant 
restrictions for current operations. 

The NASA 757 ARIES experiment was designed to achieve these objectives. Six evaluation pilots 
flew eighty-four flight test runs to Runway 25 and Runway 07 at the EGE airport. Each of the pilots 
experienced combinations of SVS display (HUD, HDD Size A, HDD Size X) and terrain texture (generic 
photo-realistic) configurations, and was asked to compare these display concepts to Baseline displays 
(i.e., EADI with TAWS/ND). Overall, the results confirm most of a priori hypotheses (the exceptions are 
indicated as well), and the significant findings of the flight test are discussed in the following sections. 

Pilot Performance 

It was hypothesized that flight technical error (in terms of RMS path performance, rather than RNP 
percentiles) would be lower for the SVS display concepts compared to the Baseline because of increased 
situation awareness and path control guidance afforded by using a flight path marker in conjunction with 
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a pathway presentation (tunnel and guidance symbol). The significant main effects for both lateral and 
vertical path control showed that pilots performed the approaches with significantly lower RMS error 
using the SVS displays compared to the Baseline display. On average, the Baseline lateral RMS error 
was 818 feet and was significantly larger than the Size A (61 ft.), Size X (51 ft.) and HUD display 
concepts (67 ft.). A similar result was found for vertical RMS error. Post-hoc analyses confirmed that 
the Baseline vertical RMS error (147 ft.) was significantly larger than the SVS displays (~37 ft.). For 
both lateral and vertical RMS error, post-hoc analyses did not find any significant differences across the 
SVS display concepts. Additionally, no significant differences were found for texture type or the 
interaction between display size and texture type. 

Analyses of RMS path performance for the individual segments, particularly for the critical final 
approach segments (segments 3 and 8), produced similar results. The most plausible reason for the 
findings are that the manually- flown approaches using the Baseline display relied on raw path error (i.e., 
“dogbone”) indicators and in some cases a dual cue flight director (only with the FMS25 approaches), 
while the SVS display concepts (in addition to the raw path error indicators) had a flight path marker and 
a pathway presentation (tunnel and guidance symbol) that provided the pilot with additional path 
guidance feedback. 

The pilot performance results described above confirm previous research (e.g., Comstock et al., 2001; 
Glaab et al., 2003) that SVS with precision guidance information can significantly reduce flight technical 
error. Similar results have also been reported in numerous pathway research studies (e.g., Williams, 
2002) confirming the advantage of making manual approaches using a tunnel because of the added 
benefits of (a) tunnel or commanded display, (b) flight path marker (i.e., velocity vector), (c) 3-D 
perspective along the pilot’s forward viewing axis, and (d) guidance (e.g., “ghost aircraft symbol”). 
Research has long established the benefits of prediction and preview (e.g., Lintern, Roscoe, & Sivier, 
1990) and presentation of this information in a 3-D perspective (Haskell & Wickens, 1993; Parrish et al., 
1994; Wickens & Prevett, 1995; Theunissen, 1997). Therefore, the finding that the SVS concepts, with 
tunnel information, yielded significantly improved pilot path performance compared to a Baseline 
concept with no tunnel was not surprising. This result suggests the use of a “pathway-in-the-sky” as a 
beneficial element in a synthetic vision system, and confirms the potential of these SVS concepts as 
retrofit candidates for replacing current displays with synthetic vision technology. 

The other objective data hypotheses dealt with anticipated FTE differences between SVS display 
features. It was anticipated that FTE performance would be directly related to the minification factor, 
with better performance being achieved as minification approached unity (i.e., performance with the HUD 
would be better than with Size X, which would be better than with Size A). While the results tend to 
support those hypotheses in the majority of cases, statistical significance was rarely obtained. Certainly 
no meaningful differences in terms of operational significance were found. The final objective data 
hypothesis dealt with texture effects, and it also was not supported by the results. Either method of 
texturing produced equivalent FTE performance, regardless of HUD or HDD implementation. The 
rejection of these hypotheses is a verification of the SVS retrofit approach. That is, HUD or HDDs of any 
size or texture method tested were equally effective means of implementing SVS concepts to achieve FTE 
benefits. 

Required Navigation Performance 

The International Civil Aviation Organization (ICAO) Special Committee of Future Air Navigation 
System (FANS) developed a new concept based in terms of communication, navigation, surveillance, and 
air traffic management (CNS/ATM). Critical to achieving the benefits of CNS/ATM concept, aircraft 


59 



will need to be able to achieve accurate, predictable, and repeatable navigation performance; this is 
termed RNP. Minimum Aviation System Performance Standards (MASPS) have been established for 
area navigation in an RNP environment, and an important objective of the EGE flight test was to establish 
actual navigation performance and compare it to RNP MASPS (RTCA, 2000). 

It was anticipated that the increased path precision provided by the tunnel presentation would enable 
pilots to make manual approaches within RNP accuracies that normally require RNAV capabilities. The 
lateral navigation analyses confirmed that flight technical error for all the SVS display concepts achieved 
an accuracy of 0.05 nmi for at least 95% of the approach compared to just 0.25 nmi for the Baseline 
condition. The vertical navigation analyses for the head-down (Size A, Size X) and head-up SVS 
concepts paralleled these results in that those concepts achieved a vertical accuracy of 150 feet and 100 
feet, respectively, at least 99.7% of the time which is better than the required vertical accuracy of 300 
feet. Vertical path control with the Baseline EADI concept (which met required accuracy 89.0% of time) 
was outside RNP permissible limits. Based on these results, therefore, synthetic vision would enable 
manual RNP operations that are one-fifth as large for lateral RNP and within required vertical 
performance accuracy values than similar operations with current 757 instruments. The outcome would 
be an increase of RNP operations to runways that otherwise would not meet current MASPS, resulting in 
a significant economic advantage to airlines employing SVS technology (Hemm, 2000; Hemm et al., 
2001). 

Pilot Preferences 

It was hypothesized that pilots would prefer all SVS display concepts over the Baseline 
EADI/Navigation Display, and that, between SVS display concepts, pilot preferences would be directly 
related to the minification factor, with preference favoring ramifications approaching unity (i.e., the HUD 
preferred over Size X, which would be preferred over Size A). The only formal assessment of pilot 
preferences utilized during the flight test was the SA-SWORD, which was actually a subjective 
assessment of the situation awareness for each of the display configurations. That assessment, which is 
discussed in detail in the next section, ranked the Size X concepts ahead of the HUD and Size A concepts, 
with the Baseline concept ranked last. Informal pilot comments also ranked the Size X concepts as the 
preferred concept, and all SVS concepts were preferred over the Baseline. Thus the hypothesis 
concerning preference favoring minifications approaching unity must be rejected. 

Situation Awareness 

To achieve the national aviation objective of reducing visibility-induced accidents, perhaps the most 
important construct that needs to be examined with SVS is how it impacts the SA of the pilot. Therefore, 
several measures of situation awareness were gathered, in addition to pilot comments. Because of the 
nature of a flight test, there are limitations on the types of measures that may be employed. Objective 
measures of SA, such as SAGAT (Endsley, 1987), do not lend themselves well to the flight research 
environment. As a consequence, subjective measures of situation awareness were used in this flight test 
including the SA-SWORD pair-comparison technique (Vidulich & Hughes, 1991) and SA run questions. 

After each test run, evaluation pilots were asked to complete a post-run questionnaire that included 
several questions about terrain awareness while flying the approaches. In general, the results suggest that 
the SVS display provided for greater terrain awareness than the Baseline EADI. Furthermore, the Size A 
photo-realistic, Size X generic, and Size X photo-realistic display concepts were rated higher in terrain 
awareness than the other SVS display concepts. Therefore, although synthetic vision was significantly 
better for pilots with regard to knowledge of terrain and aircraft position, Size A generic and the HUD 
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concepts did not afford the same level of terrain awareness. However, during the semi-structured 
interview, follow-up questions about situation awareness resulted in a non-significant finding across SVS 
display concepts. The ratings of SA (0 to 9) ranged from 6.415 (HUD generic) to 7.83 (Size A, photo- 
realistic), but were not significantly different statistically from one another. Despite representing the 
same trend in responses as the post-run questionnaire, the post-experimental debriefings showed that all 
pilots felt that each of the SVS display concepts provided adequate situation awareness and, in all cases, 
was better than the Baseline condition. 

In addition to the post-run and post-experiment questionnaires, each evaluation pilot completed the 
SA-SWORD that allows for paired comparison of each display concept for situation awareness. There 
was a significant effect found and the ranked order of the displays were: Size X Photo-realistic, Size X 
Generic, HUD Photo-realistic, Size A Photo-realistic, Size A Generic, HUD Generic, and Baseline. The 
post-hoc analyses revealed that pilots rated the Size X photo-realistic to provide greater situation 
awareness than all other SVS and Baseline concepts with the exception of Size X generic. Therefore, 
display size seems to be the primary factor determining the SA rankings, although the photo-realistic 
texture was consistently ranked higher within each display concept. Pilot comments would support this 
conclusion as many of the pilots felt that the Size X photo-realistic display concept provided the greatest 
situation awareness, but that Size X generic provided an almost equal degree of SA enhancement. All 
pilots agreed that size was more important than the texturing method although four of the pilots noted 
they would chose photo-realistic if given a choice. However, while most of the evaluation pilots did not 
discern a considerable advantage of either texture method, several pilots noted that each of the two 
texturing methods provided different information. For example, the photo-realistic texturing gave pilots 
excellent information about cultural features and was helpful for approach and departure segments. 
Generic texturing, on the other hand, showed pilots the angular relationship to the view of the terrain and 
was postulated to be useful for enroute segments. Two of the six pilots suggested that photo-realistic and 
generic should be combined into one database to take advantage of the benefits provided by both. 

The above discussion provides the validation of all of the stated a priori hypotheses concerning the 
situation awareness properties associated with size and texturing methods of the SVS concepts. More 
importantly, the hypothesis that all SVS display concepts would be rated higher in situation awareness 
than the Baseline E AD I/Navigation Display is clearly upheld. 

Workload 

Another objective of synthetic vision is to reduce mental workload of the pilot. Because SVS presents 
both natural and coded information to the pilot, it is postulated that it will both increase situation 
awareness and also reduce pilot workload by integrating often disparate pieces of information. Once 
again, the a priori hypotheses concerning workload included both that workload ratings would be 
significantly lower for all SVS display concepts compared to the Baseline EADI/Navigation Display, and 
that, within the SVS concepts, workload ratings would be directly related to the minification factor, with 
lower workload associated with minifications approaching unity (i.e., the HUD lower than Size X lower 
than Size A). 

After each test run, evaluation pilots were asked to complete a post-run questionnaire that included a 
question assessing workload experienced while flying the approaches. The results were a significant 
main effect for display type, and pilots rated the Baseline condition to be significantly higher in mental 
workload compared to the SVS concepts. No significant differences were found for either display size or 
texture type and, therefore, pilots rated the workload equally across the SVS concepts. The post-run 
questionnaire results paralleled those found for the post-flight results. Thus the hypothesis concerning 
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workload being directly related to minification factor must be rejected, while the overall SVS hypothesis 
is fully supported. 

All pilots ranked the Baseline condition to be significantly higher in workload and lower in SA than 
the SVS concepts, which were all given low workload and moderate to high SA ratings. These results 
confirm previous research (e.g., Comstock et al., 2001; Glaab et al., 2003; Stark et al., 2001) documenting 
that synthetic vision has the potential to significantly reduce workload demands and, consequently, 
increase both the safety and efficiency of aircraft operations. In addition, these results appear to indicate 
that the SVS displays evaluated are approaching the goal of true human-centered design, which is high 
situation awareness and low workload for the pilot. 

Minification Hypotheses 

Prior testing at DFW had thoroughly examined pilot selectable FOV and display size effects for 
HDDs, finding that as range to touchdown decreased, the minification factor moved toward unity (i.e., no 
minification) and that as display size increased, the minification factor also moved toward unity. 
Combining those results with the facts that the SVS HUD concept (with unity minification) had provided 
equal, if not superior, pilot/vehicle tracking performance, and superior subjective ratings, compared to the 
HDDs, led to the postulation of the hypotheses concerning enhanced performance being directly related to 
minification factor (with improvements in FTE, Pilot Preference, and Workload favoring minifications 
approaching unity). In contrast to those results obtained in nighttime testing at DFW, where the 
evaluation pilots stated that they preferred the HUD to the HDD concepts, and on which the a priori 
hypotheses concerning enhanced performance being directly related to minification factor were based, the 
EGE results rejected those hypotheses (FTE, Pilot Preference, and Workload). The rejections were based 
on HUD ratings in daylight operations. 

As at DFW, the SV-HUD concept was, for all intents, just a monochromatic green representation of 
the full-color, head-down display SV concept, using an RS-343 video format. No effort was expended to 
examine graphical light source or other terrain shading issues. In addition to the terrain presentation, the 
pathway guidance symbology was also presented in raster format (stroke presentation would have been 
preferred, but programmable stroke symbology was not available). 

As a result, two significant deficiencies were encountered: illegible display renditions under some 
direct sunlight conditions and some reported terrain depiction illusions. These deficiencies are discussed 
below. 

Pilot comments from the EGE flight trials indicated that there were instances where the sun angle 
washed out the SV HUD image and rendered the SV image unusable. To achieve the benefits of SV 
using the HUD, the SV raster image must be legible and useable in ah foreseeable ambient background 
conditions. A "useable display" can be defined using many different figures of merit, but in the case of 
the raster HUD, contrast ratio has been specified as the figure of merit. Bailey (2002) specifies that the 
HUD contrast ratio requirement for a generic or photo-realistic terrain textured SVS raster HUD 
implementation to be on the order of 5.5. This contrast ratio value was achieved only when the ambient 
brightness was below 750 ft-L. In higher ambient lighting situations, the contrast ratios were less than 
this requirement (given the finite maximum HUD brightness capability) and some raster scene details 
were washed out by the ambient light. 

Some pilots in the EGE flight test reported an occasional inversion illusion with the synthetic terrain 
HUD image, in that, at one particular point, they would interpret a valley as a ridge, and a ridge as a 
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valley. Post-flight image evaluations and experimentation with graphic light source sun angles while 
generating the monochrome terrain database appeared to eliminate the problem as discussed in Bailey 
( 2002 ). 

These two deficiencies were not apparent at DFW, and appear to have significantly affected the EGE 
results for FTE, Pilot Preference, and Workload. The FTE effects are attributed to occasional loss of 
guidance symbology, while the subjective data effects are attributed to both deficiencies. In spite of these 
deficiencies, the SVS HUD concepts performed as well as the HDD concepts all of which was vastly 
superior to present-day cockpit display technology. 

Conclusions 

The NASA Synthetic Vision Eagle-Vail flight test provided a considerable amount of valuable 
research data that have enabled crew systems researchers to significantly improve upon SVS display 
concepts. The SVS Project of Aviation Safety Program is striving to eliminate poor visibility as a causal 
factor in aircraft accidents as well as enhance operational capabilities of all aircraft through the display of 
computer generated imagery derived from an onboard database of terrain, obstacle, and airport 
information. The goal of the flight test conducted at EGE was to extend the assessment of the SVS 
retrofit approach to operations in a terrain-challenged operational environment with testing in daytime 
conditions. EGE represented an ideal location to test the effectiveness of SVS technologies for terrain 
awareness and separation for approaches and departures that put the aircraft close to mountainous terrain. 

All of the aforementioned flight test objectives were successfully achieved. The flight test was 
conducted to evaluate three SVS display types (Head-up Display, Head-Down Size A; Head-Down Size 
X) and two terrain texture methods (photo-realistic, generic) in comparison to the simulated Baseline 
Boeing-757 Electronic Attitude Direction Indicator and Navigation / Terrain Awareness and Warning 
System displays. These independent variables were evaluated for path error, situation awareness, and 
workload while making approaches to Runway 25 and 07 and during simulated engine-out Cottonwood 2 
and KREMM departures. The results of the experiment showed significantly improved performance, 
situation awareness, and workload for SVS concepts compared to the Baseline displays and confirmed the 
retrofit capability of the Head-Up Display and Size A SVS concepts. The research also demonstrated that 
the tunnel guidance display concept used within the SVS concepts achieved RNP criteria. 

Specific results of the study using objective data were: 

1 . FTE performance, both laterally and vertically, was significantly lower when using the SVS 
displays compared to the Baseline display. 

2. Within the SVS concepts, FTE performance was not directly related to the minification factor, 
with better performance being achieved as minification approached unity (i.e., performance 
with the HUD would be better than with Size X, which would be better than with Size A). 
While the results tended to support that hypothesis in the majority of cases, statistical 
significance was rarely obtained. Certainly no meaningful differences in terms of operational 
significance were found. 

3. No significant texture effects were found within the objective data. Either method of texturing 
produced equivalent FTE performance, regardless of HUD or HDD implementation. 

4. The actual navigation performance results showed that synthetic vision (HUD or HDD) would 
enable manual RNP operations that are five times smaller for lateral RNP and within required 
vertical performance accuracy values than similar operations with current 757 instruments. 
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The outcome would be an increase of RNP operations to runways that otherwise would not 
meet current MASPS, resulting in a significant economic advantage to airlines employing 
SVS technology. 

These objective data findings are a strong verification of the SVS retrofit approach. That is, HUD or 
HDDs of any size or texture method tested were an equally effective means of implementing SVS 
concepts to achieve FTE and RNP benefits. 

Specific results of the study using subjective data were: 

5. All pilots ranked the Baseline condition to be significantly higher in workload and lower in 
SA than the SVS concepts, which were all given low workload and moderate to high SA 
ratings. 

6. While larger display sizes were preferred, effective applications of SVS display technology 
can be accomplished in aircraft equipped with HDDs as small as Size-A (5.25 in. wide by 5 in. 
tall) with selectable FOV techniques. 

7. In contrast to the results obtained in nighttime testing at DFW, the EGE evaluation pilots 
stated that they preferred the HDD concepts to the HUD concept. Also the a priori hypotheses 
concerning enhanced performance being directly related to minification factor (with 
improvement favoring minifications approaching unity), which were based on the DFW 
results, were rejected by the EGE results. The rejections were based on HUD ratings in 
daylight operations. Two specific HUD deficiencies were identified, and proposed solutions 
to each have been presented. 

These subjective data results indicate that the SVS displays evaluated are approaching the goal of true 
human-centered design, which is high situation awareness and low workload for the pilot. 

The top-level results of the EGE flight test concerning the improved path performance, enhanced 
situation awareness, and lower associated workload provided by all of the SVS (HDD and HUD) 
concepts, regardless of display size, are highly significant. These results firmly establish the SVS retrofit 
concept approach as viable. 


Future Directions 

The NASA Synthetic Vision Eagle-Vail flight test provided a considerable amount of valuable 
research data that have enabled crew systems researchers to significantly improve upon SVS display 
concepts. To date, several findings of the flight test have been incorporated into development of future 
embodiments of synthetic vision. For example, several pilots suggested that photo-realistic and generic 
texturing should be combined together to achieve the best that each method has to offer, and the SVS 
Project has developed a new hybrid texture method that meets that goal. Also, the research uncovered 
several issues that were unknown before research commenced. These include the usability of raster on 
the HUD and the presentation of the “crow’s feet” minimal tunnel symbology set that was leveraged from 
earlier High-Speed Civil Transport research at NASA LaRC. Several changes have been made to resolve 
these issues, such as development of several new tunnel concepts and modifications to the HUD to render 
all symbology in stroke and render the terrain in the raster channel. 

Rockwell-Collins, a NASA industry partner, is employing a fish-net (grid) presentation of the terrain for 
their Synthetic Vision HUD concepts. The fish-net or grid presentation is a high-contrast raster image 
which should be legible throughout all ambient background luminance ranges since it mimics stroke- 
written symbology. Rockwell-Collins testing has also developed methods to ameliorate one of the past 
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problems with fish-net type displays - the annoying and distracting bright area caused by the confluence 
of edgelines in valleys or vanishing points. The United States Air Force has found an Air Force pilot 
preference for the fish-net or grid format (Snow, 2001), especially when used in combination with an 
EVS image (Rate, 1984). 

Direct comparisons between a fish-net and synthetic terrain HUD format were not conducted, but 
future NASA efforts are being directed at evaluating a fish-net terrain overlay embedded within Synthetic 
Vision terrain renditions. This approach is analogous to a fish-net synthetic terrain image combined with 
EVS. The theory is that the high contrast fish-net depiction will be noticeable and readable during all 
ambient lighting conditions, yet in lower ambient lighting conditions, the Synthetic Vision terrain 
depiction will be viewable to provide a high fidelity, unambiguous scene for terrain and obstacle 
awareness. 

Despite the progress made to address human factors research issues, research is still needed to ensure a 
“human-centered” synthetic vision system. In addition to the efforts described above, crew systems 
researchers have been actively involved in improving upon and developing new concepts. These are part 
of a suite of R&D activities that form the future directions that the SVS Project is taking. 

Previous flight tests of SVS have primarily focused on the general use and usefulness of SVS for 
providing flight critical guidance and improved situational awareness. The research objectives of these 
previous flight tests were focused on the SVS display (e.g., size, content, and format) and on SVS 
enabling technologies (e.g., Runway Incursion Prevention Systems (RIPS), Enhanced Vision Systems 
(EVS), and Database Integrity Monitoring Equipment (DIME)). While differential GPS and on-board 
databases can provide the primary framework for an operational SVS, many in the aviation community 
believe that independent integrity monitors for both surveillance and navigational functions will be 
required to meet certification and safety requirements. This functionality will rely heavily on existing on- 
board sensors (e.g., weather radars, high quality radar altimeters) to provide real-time integrity monitoring 
for the databases. Specifically, on-board integrity sensors will provide independent air-to-air, air-to- 
ground, ground-to-ground, and ground-to-air traffic and object surveillance, a runway incursion monitor 
and a confirmation of database integrity and registration (navigational position confirmation via terrain 
feature extraction). Additionally, the requirements for augmenting SVS concepts with the independent 
capabilities of weather-penetrating, enhanced vision imaging sensors during low visibility landing and 
surface operations conditions should be explored. These technologies form the basis for monitoring the 
dynamic flight environment and thereby supplement the synthetic world with real-time, direct 
measurement of the surrounding terrain and air/ground traffic. 

A flight test evaluation is anticipated in 2004 by the NASA/LaRC under NASA’s Aviation Safety, 
Synthetic Vision System Project to examine a synthetic vision system that integrates the enabling 
technologies (RIPS, EVS and DIME) of SVS. The research will focus on the integration of runway 
incursion prevention technologies, surface map displays, integrity monitoring, enhanced sensors, 
synthetic vision navigation displays, and enhanced synthetic vision primary flight and HUD displays. 
Together, such a synthetic vision system may considerably help meet national aeronautic goals to “reduce 
the fatal accident rate by a factor of 5” and to “double the capacity of the aviation system” both within 10 
years (NASA, 2001). 
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Appendix A. Synthetic Vision Systems Project Background 

Background 

In August 1996, following the wake of several commercial transport accidents that raised the level of 
public awareness, a White House Commission on Aviation Safety and Security was established to study 
matters involving these pertinent issues. The Commission’s findings (Department of Transportation, 
1997) concluded that although the worldwide commercial aviation major accident rate is low and has 
been nearly constant over the past two decades, increasing traffic over the years has resulted in an 
increase in the absolute number of accidents. The demand for air travel is expected to increase over the 
coming two decades, more than doubling by 2017. Without an improvement in the accident rate, such an 
increase in traffic volume would lead to a projected 50 or more major accidents a year worldwide - a near 
weekly occurrence. Given the very tragic, and damaging effects of a single major accident, this situation 
could become an unacceptable blow to the public’s confidence in the aviation system. As a result, the 
anticipated growth of the commercial air-travel market would not reach its full potential. In February 
1997, in response to the Commission’s recommendations, President Clinton set a national goal to reduce 
the aviation fatal accident rate by 80% within ten years. 

NASA Agency Role 

A high priority national challenge is to ensure U.S. leadership in aviation in the face of growing air 
traffic volume, new safety requirements, and increasingly stringent noise and emissions standards. NASA 
has a successful history of leading the development of aggressive high payoff technology in high-risk 
areas, ensuring a proactive approach is taken to developing technology that will both be required for 
meeting anticipated future requirements, and for providing the technical basis to guide policy by 
determining feasible technical limits. Therefore, NASA created the Aviation Safety Program (AvSP) to 
address the President’s national aviation safety goal. NASA sponsored a major program planning effort 
to gather input from the aviation community regarding the appropriate research to be conducted by the 
Agency. The NASA Aviation Safety Investment Strategy Team (ASIST) held four industry- and 
government-wide workshops to define and recommend research areas which would have the greatest 
potential impact for reducing the fatal accident rate. NASA then redirected existing research and 
technology efforts and formulated new ones to address the safety needs defined by ASIST (Norman, 
2001 ). 

One of the significant recommendations from ASIST was to establish a project to eliminate visibility- 
induced errors for all aircraft through the cost-effective use of synthetic/enhanced vision displays, 
worldwide terrain databases, and global positioning system (GPS) navigation. Therefore, the Associate 
Administrator for Aerospace Technology, Spence Armstrong, signed the Project Formulation 
Authorization for the Synthetic Vision Systems Project. 

NASA Synthetic Vision Research Project 

NASA stepped up to the challenge of eliminating visibility-induced accidents by forming the AvSP 
Synthetic Vision Systems (SVS) Project. Limited visibility is the single most critical factor affecting both 
the safety and capacity of worldwide aviation operations. In commercial aviation alone, over 30-percent 
of all fatal accidents worldwide (Boeing, 1996) are categorized as controlled flight into terrain (CFIT), 
where a mechanically sound and normally functioning airplane is inadvertently flown into the ground, 
water, or an obstacle, principally due to the lack of outside visual reference and situation awareness (SA). 
Another type of accident involving restricted visibility combined with compromised situational awareness 
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is runway incursions. The AvSP SVS Project is developing technologies with practical applications that 
will eliminate low visibility conditions as a causal factor to civil aircraft accidents, as well as replicate the 
operational benefits of flight operations in unlimited ceiling and visibility day conditions, regardless of 
the actual outside weather or lighting condition. The technologies will emphasize the cost-effective use 
of synthetic/enhanced- vision displays; worldwide navigation, terrain, obstruction, and airport databases; 
and GPS-derived navigation to eliminate “visibility-induced” (lack of visibility) errors for all aircraft 
categories. A major thrust of the SVS Project is to develop and demonstrate affordable, certifiable 
display configurations which provide intuitive out-the- window terrain and obstacle information, including 
guidance information for precision navigation and obstacle/obstruction avoidance for commercial and 
business aircraft. 

The ultimate goal of the SVS Project is to eliminate low visibility as a causal factor of civil aircraft 
accidents as recommended by the ASIST team, which would significantly help achieve national aviation 
safety goals. In addition, SVS may increase the efficiency of the National Airspace System by allowing 
precision Instrument Meteorological Conditions (IMC) operations, which presently require extensive 
ground infra-structure, such as Instrument Landing Systems (ILS), to many more runways than are 
permitted today by providing safer operations and lower weather minimums (i.e., Category I, II, Ilia, Illb) 
for landing at non-ILS-equipped airports. 

The SVS Project is taking the approach of employing a visual-based solution to overcome reduced 
pilot situation awareness caused by limited outside visibility. As a part of the SVS Project, the Synthetic 
Vision Display Concepts (SVDC) group focuses on SVS applications for commercial and business 
aircraft by designing, developing and implementing SVS display concepts for flight test and simulation 
evaluations and by conducting subsequent research activities. These SVS displays will provide the pilot 
with a clear view of the outside world through the application of sensors, such as Forward-Looking Infra- 
Red (FLIR), Radar, and Millimeter- Wave technologies; navigation and terrain databases; and 
computational subsystem components, such as image object detection and symbology generation. 
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Appendix B. Theoretical Foundations of Synthetic Vision Systems 


It is highly unlikely that conventional display concepts can significantly increase safety as new 
functionality and new technology cannot simply be layered onto previous design concepts since the 
current system complexities are already too high (Theunissen, 1997). Better human- machine interfaces 
require a fundamentally new approach. One such approach applies the fundamental advantage of 
perspective flight path displays relative to conventional displays. Rather than commanding the pilot what 
to do, or at best showing only the error with respect to the desired trajectory, guidance and navigation 
displays can now provide information about the margins within which the pilot is allowed to operate. 
These displays are augmented to show such information as spatial constraints and terrain constraints, 
rather than just showing conventional flight director commands, which only indicate an error, seemingly 
independent of the actual constraints. These additional display elements provide guidance that does not 
force the pilot to apply a continuous compensatory control strategy. Only in this way can human 
flexibility be exploited. This is a fundamental difference between current and synthetic vision systems 
(SVS) displays - that synthetic vision embodies the concept of “human-centered” design by providing 
natural versus coded information to the pilot (Theunissen, 1997). 

Human-Centered SVS Displays. The term, “human-centered”, is used to define an approach that 
designs to accommodate the human user in contrast to the more common “technology-centered” 
approach. The rapid advance of technology in the cockpit has had an unintended consequence of 
isolating the pilots and decreasing their situation awareness by increasing systems complexity, reducing 
crew-vehicle coupling, enhancing system autonomy, and reducing systems feedback (Billings, 1997). 
Essentially, layers of technology have removed the pilots from aircraft control, leading to the “out-of-the- 
loop” problem. A number of researchers have proposed a set of principles of human-centered design. 
Billings (1997) offered the following that are relevant to SVS: 

Premise: The pilot bears the responsibility for safety of flight 

Axiom: Pilots must remain in command of their flights 

Corollaries: Pilots must be actively involved 

Pilots must be adequately informed 

Pilots must be able to monitor the system assisting them 

The use of the human-centered design perspective has important implications for the reduction of 
visibility-induced accidents. Past technologies have been developed with the intended purpose of 
prevention of several accident categories attributed to low- visibility. For example, the Ground Proximity 
Warning System (GPWS) was introduced in 1973 and, despite initial problems with a high false alarm 
rate, the 10-30 second “look ahead” capability of the system has significantly improved the situation 
awareness (SA) of flightcrews. However, numerous accidents, such as the A-300 Thai Airlines (1992) 
and B-757 American Airlines (1995) accidents, show that the GPWS is not the final solution. The 
introduction of the Enhanced GPWS (EGPWS) has mitigated some of the “misuse” and “disuse” 
(Parasuraman & Riley, 1997) issues that confront GPWS. It provides for more warning time - up to 60 
seconds - and takes advantage of a worldwide digital terrain database and a color-coded display of the 
surrounding terrain. Ladkin (1997) asserted that there is near unanimity of the acceptance that EGPWS 
has improved aviation safety and reduced the incidences of controlled flight into terrain (CFIT). 
However, the use of technology generally follows the “warn-act” model and, therefore, requires the 
flightcrew to be reactive rather than proactive. The technology provides a warning when theoretically the 
flightcrew has already lost spatial and situation awareness and must then perform an escape maneuver. 
As Moroze et al. (1999) describe, the strategy may not be optimal given the reaction times required to 
initiate the escape maneuver and the cognitive and naturalistic decision making constraints required to 
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adequately encode and assess the situation (i.e., situation assessment and action implementation; 
Parasuraman, Sheridan, & Wickens, 2000). Essentially, then, EGPWS is a “warning system” and doesn’t 
support a human-centered design philosophy in the objective of reduction of CFIT accidents. Snow et al. 
(1999) declared that what is needed is an intuitive system that improves pilot situation awareness with 
respect to spatial orientation in terms of terrain and flight path, and does not require the pilot to divert 
visual attention and cognitive resources away from possible external events and primary flight reference; 
that is, to provide a human-centered technology that can help prevent rather than just inform the 
flightcrews of a potential collision with terrain. The approach requires an understanding and exploitation 
of the unique information processing capability of flight crews and a design of the technology and 
interface to accommodate perceptual and cognitive capabilities of the pilots - the difference between a 
“natural” and a “coded” display. 

Theunissen (1997) discussed the concept of natural versus coded information. Natural information 
implies that the method of information acquisition by the pilot is similar to that experienced in Visual 
Meteorological Conditions (VMC) by looking out the window. Visual altitude judgment is an example of 
natural information acquisition. Coded information implies some type of information presentation to the 
pilot that requires interpretation to comprehend the actual value. An example of coded information is 
digital radio altitude. Theunissen noted that it is very important to give the pilot information required to 
maintain SA in low-visibility conditions and that natural information presentation is intuitive and able to 
be perceived in a much more rapid manner than coded information. SVS displays provide a natural 
presentation of the outside world with information that is intuitive and easy to process. 
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Appendix C: Situation Awareness and Workload in Relation to Synthetic 
Vision Systems 

Situation Awareness and SVS. There are numerous definitions of situation awareness (SA) and what 
it best represents. Sarter and Woods (1991) defined it as, “accessibility of a comprehensive and coherent 
situation representation which is continuously being updated in accordance with the results of the 
recurrent situation assessments.” Vidulich (1994) defined SA as, “the capability to appropriately assess 
yourself, your system, and your environment in order to make the right decision at the right time.” 
Endsley (1988), in contrast, stated that SA is comprised of three levels: “the perception of the elements of 
the environment within a volume of time and space [Level 1], the comprehension of their meaning [Level 
2], and the projection of their status in the near future [Level 3].” Because there are so many definitions 
of what SA is, Wickens (1995) offered up a consensus definition which he proposes as: “Situation 
awareness is the continuous extraction of environmental information about a system or environment, the 
integration of the information with previous knowledge to form a coherent mental picture, and the use of 
that picture in directing further perception, anticipating and responding to future events.” A synthetic 
vision system, therefore, has significant potential to improve situation awareness by fostering and 
developing a “picture”, both figuratively and literally, of the environment and presenting that picture so 
that a pilot could “stay ahead” of the aircraft and maintain an accurate mental model. 

Workload and SVS. Workload and situation awareness are related but are distinct psychological 
constructs and, therefore, have different theoretical implications for the design of a synthetic vision 
systems (SVS) display. Endsley (1988) noted that SA and workload could be differentiated through the 
effects on the pilot due to variations in task, system operation, or individual differences. She described 
that there are four extremes in which there is a “dissociation” between the two measures: 

> low SA and low workload can arise because of inattentiveness and low motivation; 

> low SA and high workload can lead to a loss of SA because the volume of information and 
task demands tax the ability of the pilot to keep up with the current information and properly 
analyze data significance; 

Ip? high SA and high workload often confronts flight crews wherein the task demands are great 
but the pilots work hard to manage the task situation; and 

> high SA and low workload wherein flight crews are provided with the right information at the 
right time and in the right amount - the goal of true human-centered design and one that an 
SVS display as part of a human factored system could engender. 

Definitions of workload are as numerous as definitions of situation awareness. Davis (1957) stated 
that, "paying attention to an object or an activity is regarded as imposing a measurable cost upon limited 
information processing resources which varies with the amount of attention paid, or the degree of 
‘intensive’ attention. The processing of attention by the nervous system depends first upon the quality of 
the stimulus input, second upon the availability of mental structures to perform the mental operations 
necessary for processing the input, and third upon a supply of mental resources or capacity which 
provides the energy required for those operations to be carried out." There are a number of theories (e.g., 
confusion theory, single undifferentiated capacity theory, multiple resource theory) and measurement 
techniques (i.e., primary, secondary, physiological, and subjective) of workload. 

The difference between the amount of cognitive resources available to perform a task and the 
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difficulty of the task determines how much mental workload the pilot or flight crew experiences. Gopher 
et al. (1986) stated that, “Mental workload may be viewed as the difference between the capacities of the 
information processing system that are required for task performance to satisfy performance expectations 
and the capacity available at a given time.” The construct of mental workload, therefore, is partly 
“hardware” (e.g., channel capacity) and partly “software” (e.g., situational cognitive appraisal). Pilots are 
constantly confronted with new information that they have to recognize, encode, synthesize, decide upon 
proper course of actions to take, and then implement the action sequence to effectuate the activated 
decision node. Miller (1956) and information theory literature show that the channel bandwidth is limited 
to 5 - 9 “chunks” of information. There are considerable individual differences amongst pilots in 
strategies used, experiences, skills, motivations, responsiveness, and cognitive and resource management 
abilities. These differences can determine what constitutes a "chunk" (e.g., Miller, 1956) and whether the 
collective resources of the flight crew will be taxed, leading to increased mental workload. As the pilots’ 
capabilities to manage the task situation are exceeded, the mental workload induced can lead to 
performance deficits. Furthermore, research has shown that as mental workload increases, task 
saturation, peripheralization, tunneling, and other outcomes significantly hinder a flight crews’ ability to 
adequately do situation assessment, cross-checking, and hypothesis generation. Flight crews, confronted 
with high mental workload situations, often get “bottled in” to a single solution and fail to be able to “stay 
ahead of the aircraft” (i.e., Level 111 situation awareness; see Endsley, 1987) and perform projective 
management of the requirements to aviate, navigate, and communicate (Endsley, 1988; Klein, 1993; 
Jensen, 1995). Often, the flight crews get “led down the garden path” and get further and further behind 
the aircraft until they can no longer manage the task environment and understand state events, putting the 
aircraft into a dangerous and sometimes unrecoverable situation. SVS, therefore, could significantly 
mitigate a high workload situation by providing information in a natural, intuitive format that doesn’t tax 
the pilots in the requirement to integrate disparate pieces of flight / state data to develop and update their 
mental model. Presentation of precision guidance that is overlaid on a perspective, 3-D SVS display 
could significantly enhance situation awareness and free the cognitive resources of the flight crew to 
manage the non-normal or emergency events, compared to having to synthesize information from the 
speed and altitude tapes, flight director, terrain depiction / color codes on the navigation display, flight 
path and constraints, aircraft performance capabilities. Mode Control Panel settings, etc. 
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Appendix D: Required Navigation Performance 


Required navigation performance (RNP) is a statement of the navigation performance accuracy 
necessary for operation within a defined airspace. RNP airspace is a generic term referring to airspace, 
routes, and legs, where minimum navigation performance requirements have been established and aircraft 
must meet or exceed that performance to fly in that airspace. The system performance requirements for 
RNP Area Navigation (RNAV) is that each aircraft operating in RNP airspace shall have total system 
error components in the cross-track and along-track directions that are less than the RNP value 95% of the 
flying time. RNP type is a designator according to navigational performance accuracy in the horizontal 
plane (lateral and longitudinal position fixing). This designator invokes all of the navigation performance 
requirements associated with the applicable RNP number, which is a containment value. For example, 
RNP-1 means that for at least 95% of the time the navigational performance in the horizontal plane, or the 
total horizontal system error, is less than 1.0 nautical mile (nmi). In addition to requiring 95% positioning 
accuracy for RNP operations, these types of procedures also require integrity of the positioning accuracy 
at 99.999% at 2 x RNP number. In our example above with an RNP-1, the position accuracy within 2.0 
nmi of the ownship (2 x RNP value of 1.0 nmi) would have to be guaranteed to be correct 99.999% of the 
time to enable RNP-1 operations. 

There are three lateral components of navigation error: path definition error, path steering error, and 
position estimation error (RTCA, 2000). These errors, defined in the following, represent the total 
horizontal system error of the airplane and are the difference between the aircraft’s true position and 
desired position (see fig. Dl): 

> The path definition error is the difference between the defined path and the desired path at a 
specific point. 

> The path steering error is the distance from the estimated position to the defined path. It 
includes both the flight technical error (FTE) and display error. FTE is the accuracy with 
which the aircraft is controlled as measured by the indicated aircraft position with respect to 
the indicated command or desired position. 

> The position estimation error, also referred to as the ship’s actual navigation performance 
(ANP), is the difference between the true position and the estimated position. 


Desired Path 


Path Definition Error 


Defined Path 


True Position 


Estimated Position 




Figure Dl. Lateral components of navigation error terms 
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Vertical navigation (VNAV) capability further enhances flight operations by enabling the specification 
of a flight path vertically for the lateral flight path. The system performance requirements for VNAV are 
that for at least 99.7% of the time the navigational performance in the vertical plane, or the total vertical 
system error, is less than a specified altitude deviation measure based on the airspace being flown in 
(below 5000 feet MSL, 5000-10000 feet MSL, above 10000 feet MSL) and the type of flight operation 
(level flight/c limb/descent or flight along specified vertical profile) being performed (see table Dl). 

There are four vertical components of navigation error: altimetry system error, vertical path steering 
error, vertical path definition error, and horizontal coupling error (RTCA, 2000). These errors, defined in 
the following, represent the total vertical system error of the airplane and are the difference between the 
aircraft’s true vertical position and desired vertical position at the true lateral position (see fig. D2): 

> Altimetry system error is the error attributable to the aircraft altimetry installation, including 
position effects resulting from normal aircraft flight attitudes. 

> The vertical path steering error is the distance from the estimated vertical position to the 
defined path. It includes both FTE and display error. 

> The vertical path definition error is the vertical difference between the defined path and the 
desired path at the estimated lateral position. 

> The horizontal coupling error is the vertical error resulting from horizontal along track 
position estimation error coupling through the desired path. 


Table Dl. Vertical Accuracy Performance Requirements 


Error Source 

Level Flight Segments 
and Climb/Descent 
Intercept of Clearance 
Altitudes (MSL) 

Approach 
along Specified 
Vertical Profile 
(MSL) 


At or Below 
5000 ft 

5000 ft to 
10000 ft 

Above 
10000 ft 

At or Below 
5000 ft 

5000 ft to 
10000 ft 

Above 10000 ft 

Altimetry 

90 ft 

200 ft 

250 ft 

140 ft 

265 ft 

350 ft 

RNAV 

Equipment 

50 ft 

50 ft 

50 ft 

100 ft 

150 ft 

220 ft 

Flight 

Technical 

150 ft 

240 ft 

240 ft 

200 ft 

300 ft 

300 ft 

Total Root- 

Sum-Square 

(RSS) 

190 ft 

320 ft 

350 ft 

265 ft 

430 ft 

510 ft 1 
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True Position 



Defined Path 


Desired Path 
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Appendix E. Planned Run Matrix 


The following tables show the planned run matrix / configurations by pilot for the 12 sorties (data 
collection flights). 



Vision 

Restriction 


Pilot # 





Sortie # 


1 


Run # 


1 

2 

3 

4 

5 

6 


1 

2 

3 

4 

5 

6 


Display 


25 

none 

Baseline with flight director 

none 

25 

none 

HUD, Photo-texture 

yes 

25 

yes 

HUD, Photo-texture 

yes 

25 

yes 

HUD, Generic-texture 

yes 

7 

none 

Baseline with flight director 

none 

7 

yes 

Industry Partner 

yes 

25 

yes 

Industry Partner 

yes 

25 

yes 

Size A, Generic-texture 

yes 

25 

yes 

Size A, Photo-texture 

yes 

7 

yes 

HUD, Photo-texture 

yes 

7 

yes 

Size A, Photo-texture 

yes 

7 

yes 

Size X, Generic-texture 

yes 



3 

1 












2 













25 

none 

Baseline with flight director 

none 

25 

none 

HUD, Generic-texture 

yes 

25 

yes 

HUD, Generic-texture 

yes 

25 

yes 

HUD, Photo-texture 

yes 

7 

none 

Baseline with flight director 

none 

7 

yes 

Industry Partner 

yes 

25 

yes 

Industry Partner 

yes 

25 

yes 

Size A, Photo-texture 

yes 

25 

yes 

Size A, Generic-texture 

yes 

7 

yes 

HUD, Photo-texture 

yes 

7 

yes 

Size X, Photo-texture 

yes 

7 

yes 

Size A, Generic-texture 

yes 



25 

none 

Baseline with flight director 

none 

25 

none 

HUD, Photo-texture 

yes 

25 

yes 

HUD, Photo-texture 

yes 

25 

yes 

HUD, Generic-texture 

yes 

7 

none 

Baseline with flight director 

none 

7 

yes 

Industry Partner 

yes 

25 

yes 

Industry Partner 

yes 

25 

yes 

Size X, Generic-texture 

yes 

25 

yes 

Size X, Photo-texture 

yes 

7 

yes 

HUD, Generic-texture 

yes 

7 

yes 

Size A, Generic-texture 

yes 

7 

yes 

Size X, Photo-texture 

yes 
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Vision 

Restriction 

Device 

Display 

Condition 

Pathway 

none 

Baseline with flight director 

none 

none 

HUD, Generic-texture 

yes 

yes 

HUD, Generic-texture 

yes 

yes 

HUD, Photo-texture 

yes 

none 

Baseline with flight director 

none 

yes 

Industry Partner 

yes 

yes 

Industry Partner 

yes 

yes 

Size X, Photo-texture 

yes 

yes 

Size X, Generic-texture 

yes 

yes 

HUD, Generic-texture 

yes 

yes 

Size X, Generic-texture 

yes 

yes 

Size A, Photo-texture 

yes 


none 

Baseline with flight director 

none 

none 

HUD, Photo-texture 

yes 

yes 

HUD, Photo-texture 

yes 

yes 

HUD, Generic-texture 

yes 

none 

Baseline with flight director 

none 

yes 

Industry Partner 

yes 

yes 

Industry Partner 

yes 

yes 

Size A, Photo-texture 

yes 

yes 

Size A, Generic-texture 

yes 

yes 

HUD, Photo-texture 

yes 

yes 

Size A, Generic-texture 

yes 

yes 

Size X, Photo-texture 

yes 


none 

Baseline with flight director 

none 

none 

HUD, Generic-texture 

yes 

yes 

HUD, Generic-texture 

yes 

yes 

HUD, Photo-texture 

yes 

none 

Baseline with flight director 

none 

yes 

Industry Partner 

yes 

yes 

Industry Partner 

yes 

yes 

Size X, Generic-texture 

yes 

yes 

Size X, Photo-texture 

yes 

yes 

HUD, Generic-texture 

yes 

yes 

Size X, Generic-texture 

yes 

yes 

Size A, Photo-texture 

yes 
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Appendix F. Post-Flight Questionnaire Ratings 

In the text below, the symbol, %, represents the mean and the symbol, a, represents one standard 
deviation. These two symbols are highlighted, italicized, and in bold font, for each question that 
was analyzed and presented in the Results/Subjective Data Analyses section of this paper. 

Baseline EADI w/ flight director 

1 . Please rate the ease of performing the approach to rwy. 25 using the traditional EADI with 
flight director 


EASE OF USE TO RWY. 25 USING EADI 


H 

1 

6 7 8 

9 


Very Hard Somewhat Hard 

Somewhat Easy 

Very Easy 


Pilot 1: 5 

Pilot 4: 3 

% = 

6.16 

Pilot 2: 7 

Pilot 5: 7 

G = 

2.04 

Pilot 3: 9 

Pilot 6: 6 




2. Please rate the ease of performing the approach to rwy. 7 using the traditional EADI with flight 
director 


EASE OF USE TO RWY. 7 USING EADI 


6 7 8 

9 

Very Hard Somewhat Hard 

Somewhat Easy 

Very Easy 


Pilot 4:1 % = 5.16 

Pilot 5: 4 c = 2.71 

Pilot 6: 5 

NASA SVS HUD 


Pilot 1: 5 
Pilot 2: 7 
Pilot 3: 9 


3. Evaluate the ease of using the HUD photo-texture during the approach to rwy. 25 


EASE OF USE OF HUD PHOTO TO RWY. 25 


6 7 8 

9 

Very Hard Somewhat Hard 

Somewhat Easy 

Very Easy 


Pilot 1: 7 
Pilot 2: 8 
Pilot 3: 9 


Pilot 4: 7 
Pilot 5: 7 
Pilot 6: 7 


X = 7.5 
a = 0.83 
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4. Evaluate the ease of using the HUD generic-texture during the approach to rwy. 25 


EASE OF USE OF HUD GENERIC TO RWY. 25 



6 7 8 

9 


Very Hard Somewhat Hard 

Somewhat Easy 

Very Easy 


Pilot 1: 5 

Pilot 4: 7 

X = 

6.66 

Pilot 2: 8 

Pilot 5: 7 

c = 

1.86 

Pilot 3: 9 

Pilot 6 : 4 




5. Evaluate the ease of using the HUD photo-texture during the approach to rwy. 7 


EASE OF USE OF HUD PHOTO TO RWY. 7 



6 7 8 

9 


Very Hard Somewhat Hard 

Somewhat Easy 

Very Easy 


Pilot 1: 3 

Pilot 4: 7 

2 C = 

6.83 

Pilot 2: 8 

Pilot 5: 7 

G - 

2.04 

Pilot 3: 9 

Pilot 6 : 7 




6 . Evaluate the ease of using the HUD generic-texture during the approach to rwy. 7 


EASE OF USE OF HUD GENERIC TO RWY. 7 


6 7 8 

9 

Very Hard Somewhat Hard 

Somewhat Easy 

Very Easy 


Pilot 1: 6 Pilot 4: 7 % = 6.83 

Pilot 2: 8 Pilot 5:1 c = 1.12 

Pilot 3: 9 Pilot 6 : 4 

7. Evaluate the ease of performing a missed approach to rwy. 25 using the HUD generic -texture 
display concept 


EASE OF USE OF HUD GENERIC TO MISSED APPROACH RWY. 25 





Very Hard 

Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:5 Pilot 4: 7 ^ = 6.5 

Pilot 2: 7 Pilot 5: 7 0 - | .76 

Pilot 3: 9 Pilot 6 : 4 
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You responded that the ease of using the HUD generic -texture during the approach to rwy 7 was 

during your first day of flight evaluation. Please comment on the perceived 

differences (in rating if there is one) between using the generic-texture HUD to rwy. 7 and rwy. 


25. 


Pilot 1 : No differences 
Pilot 2: No differences 
Pilot 3: No differences 
Pilot 4: No differences 
Pilot 5: No differences 
Pilot 6: Generic harder 

8. Please rate the workload associated between the two different approaches. We can define 
workload as: “the degree of cognitive processing capacity required to perform the flight task 
approach adequately”. Please rate the workload in using the HUD generic -texture during the 
approach to rwy. 7 


WORKLOAD TO RWY. 7 USING HUDGENERIC 


1 

2 

3 

4 

5 

6 

7 

8 


Very High Somewhat High Somewhat Low Very Low 


Pilot 1 : 5 Pilot 4:8 % = 6.33 

Pilot 2: 7 Pilot 5: 6 <j= 1,03 

Pilot 3: 6 Pilot 6: 6 

9. Please rate the workload in using the HUD generic -texture during the approach to rwy. 25 


WORKLOAD TO RWY. 25 USING HUD GENERIC 


2 

3 

4 

5 

6 

7 

OO 


Very High Somewhat High Somewhat Low Very Low 


Pilot 1 : 5 Pilot 4: 7 % = 6.0 

Pilot 2: 7 Pilot 5: 5 <j = 0.89 

Pilot 3: 6 Pilot 6: 6 

10. Please rate the missed approach, rwy. 7 or rwy. 25, which was lower in workload with using 
the HUD generic-texture. rwy. 7 rwy. 25 

Pilot 1 : Equal 
Pilot 2: Equal 
Pilot 3: Equal 
Pilot 4: Equal 
Pilot 5: Equal 
Pilot 6: Equal 
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1 1 . Please evaluate your situation awareness during the two different approaches to rwy. 25 using 
the NASA SV HUD generic-texture display concept. We define situation awareness as: “...the 
pilot has an integrated understanding of the factors that will contribute to the safe flying of the 
aircraft under normal or non-normal conditions.” 


SA TO RWY. 25 USING HUD GENERIC 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1: 5 Pilot 4: 8 6.5 

Pilot 2: 8 Pilot 5: 6 <j= ],74 

Pilot 3: 8 Pilot 6: 4 

12. Please evaluate your situation awareness during the two different approaches to rwy. 7 using 
the NASA SV HUD generic-texture display concept. We define situation awareness as: “...the 
pilot has an integrated understanding of the factors that will contribute to the safe flying of the 
aircraft under normal or non-normal conditions.” 


SA TO RWY. 7 USING HUD GENERIC 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1: 5 Pilot 4: 8 %=6.33 

Pilot 2: 8 Pilot 5: 5 (j- J,S6 

Pilot 3: 8 Pilot 6: 4 

13. Please evaluate your situation awareness during the approach to rwy. 25 using the NASA SV 
HUD photo-texture display concept. We define situation awareness as: “...the pilot has an 
integrated understanding of the factors that will contribute to the safe flying of the aircraft under 
normal or non-normal conditions.” 


SA TO RWY. 25 USING HUD PHOTO 


2 

3 

4 

5 

6 

7 

OO 


Very Low Somewhat Low Somewhat High Very High 


Pilot 1: 

6 

Pilot 4: 8 

X = 6.83 

Pilot 2: 

8 

Pilot 5: 5 

a =1.33 

Pilot 3: 

8 

Pilot 6: 6 
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14. Please evaluate your situation awareness during the approach to rwy. 7 using the NASA SV 
HUD photo-texture display concept. We define situation awareness as: “...the pilot has an 
integrated understanding of the factors that will contribute to the safe flying of the aircraft under 
normal or non-normal conditions.” 


SA TO RWY. 7 USING HUD PHOTO 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1: 6 Pilot 4: 8 ^=6.66 

Pilot 2: 8 Pilot 5: 5 <j= 1.S0 

Pilot 3: 8 Pilot 6: 5 

15. Did the use of the NASA HUD significantly improve your situation awareness beyond the use 
of the EADI w/ flight director for approach to rwy. 25? Please rate how much more your level of 
situation awareness was enhanced. 


SA IMPROVEMENT USING HUD OVER EADI Rwy. 25 

1 

2 

3 4 5 6 7 8 9 

None 


A Little Somewhat Significantly 


Pilot 1:6 Pilot 4: 9 % = 7.83 

Pilot 2: 7 Pilot 5: 7 a =1.33 

Pilot 3: 9 Pilot 6: 9 

16. Did the use of the NASA HUD significantly improve your situation awareness beyond the use 
of the EADI w/ flight director for approach to rwy. 7? Please rate how much more your level of 
situation awareness was enhanced. 


SA IMPROVEMENT USING HUD OVER EADI Rwy. 7 

1 

2 


None 


A Little Somewhat Significantly 


Pilot 1: 5 

Pilot 

Pilot 2: 8 

Pilot 

Pilot 3: 9 

Pilot 


X - 7-83 

a = 1.60 
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17. Evaluate the ease of using the HUD generic-texture during the missed approach to rwy. 7 


EASE OF USE OF HUD GENERIC TO MISSED APPROACH RWY. 7 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard 


Somewhat Hard 


Somewhat Easy 


Very Easy 


Pilot 1 : 5 Pilot 4: 7 % = 6.5 

Pilot 2: 7 Pilot 5: 7 0=1.76 

Pilot 3: 9 Pilot 6: 4 

18. Evaluate the ease of performing a missed approach to rwy. 7 using the HUD photo-texture 
display concept 


EASE OF USE OF HUD PHOTO TO MISSED APPROACH RWY. 7 


1 


Very Hard 


Somewhat Hard 


6 7 


Somewhat Easy 


Very Easy 


Pilot 1: 5 Pilot 4: 7 % = 6.5 

Pilot 2: 7 Pilot 5: 7 0=1.76 

Pilot 3: 9 Pilot 6: 4 

19. Please rate which NASA HUD display concept (e.g., generic) provided the best level of 
situation awareness in performing the missed approach to rwy. 7 (situation awareness defined as: 
“. . .the pilot has an integrated understanding of the factors that will contribute to the safe flying of 
the aircraft under normal or non-normal conditions.”). 

Pilot 1 : Neither 
Pilot 2: Photo 
Pilot 3: Photo 
Pilot 4: Photo 
Pilot 5: Photo 
Pilot 6: Photo 

NASA HUD Symbology 

20. Evaluate the ease of predicting flight path using the “follow-me aircraft”: 


EASE OF PREDICTING FLIGHT PATH 



6 7 8 

9 


Very Hard Somewhat Hard 

Somewhat Easy 

Very Easy 


Pilot 1: 7 

Pilot 4: 8 

% = 

7.16 

Pilot 2: 7 

Pilot 5: 8 

0 = 

0.75 

Pilot 3: 6 

Pilot 6: 7 
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2 1 . Evaluate the ease of using the tunnel for vertical flight path guidance 


EASE OF TUNNEL FOR VERTICAL FLIGHT PATH GUIDANCE 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1 : 7 Pilot 4: 7 % = 7. 16 

Pilot 2: 7 Pilot 5: 8 a =1.33 

Pilot 3: 9 Pilot 6: 5 

22. Evaluate the ease of using the tunnel for lateral flight path guidance 


EASE OF TUNNEL FOR LATERAL FLIGHT PATH GUIDANCE 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 4: 7 i - 7.0 

Pilot 5: 7 a =1.26 

Pilot 6: 5 

Size A Evaluation 


Pilot 1: 7 
Pilot 2: 7 
Pilot 3: 9 


23. Evaluate the ease of interpreting airspeed information for Size A photo: 


EASE OF INTERPRETING AIRSPEED INFORMATION SIZE A PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 8 Pilot 4: 8 % = 7.66 

Pilot 2: 9 Pilot 5: 9 c=1.36 

Pilot 3: 6 Pilot 6: 6 

24. Evaluate the ease of interpreting airspeed information for Size A generic: 


EASE OF INTERPRETING AIRSPEED INFORMATION SIZE A GENERIC 


2 

3 

4 

5 

6 

7 

OO 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 8 
Pilot 2: 9 
Pilot 3: 8 


Pilot 4: 8 'i = 8.0 

Pilot 5: 9 a =1.09 

Pilot 6: 6 
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25. Evaluate the ease of interpreting flight path vectors for Size A photo: 


EASE OF INTERPRETING FLIGHT PATH VECTORS SIZE A PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 8 Pilot 4: 9 % = 8.16 

Pilot 2: 8 Pilot 5:8 & = 0.41 

Pilot 3: 8 Pilot 6: 8 

26. Evaluate the ease of interpreting flight path vectors for Size A generic: 


EASE OF INTERPRETING FLIGHT PATH VECTORS SIZE A GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:8 Pilot 4: 9 1-1.61 

Pilot 2: 8 Pilot 5: 8 a = 1.36 

Pilot 3: 8 Pilot 6: 5 

27. Evaluate the ease of interpreting altitude information for Size A photo: 


EASE OF INTERPRETING ALTITUDE INFORMATION SIZE A PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:8 Pilot 4: 8 Z = 7.33 

Pilot 2: 9 Pilot 5: 8 c=1.51 

Pilot 3: 6 Pilot 6: 5 

28. Evaluate the ease of interpreting altitude information for Size A generic: 


EASE OF INTERPRETING ALTITUDE INFORMATION SIZE A GENERIC 


2 

3 

4 

5 

6 

7 

OO 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 8 
Pilot 2: 9 
Pilot 3: 6 


Pilot 4: 8 x = 7-33 

Pilot 5: 8 a =1.51 

Pilot 6: 5 
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29. Evaluate the ease of interpreting vertical speed information for Size A photo: 


EASE OF INTERPRETING VERTICAL SPEED INFORMATION SIZE A PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1 : 8 Pilot 4:8 % = 7.0 

Pilot 2: 8 Pilot 5: 7 a =1.26 

Pilot 3: 6 Pilot 6: 5 

30. Evaluate the ease of interpreting vertical speed information for Size A generic: 


EASE OF INTERPRETING VERTICAL SPEED INFORMATION SIZE A GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1 : 8 Pilot 4:8 % = 7.0 

Pilot 2: 8 Pilot 5: 7 a = 1.26 

Pilot 3: 6 Pilot 6: 5 

3 1 . Evaluate the ease of interpreting the ILS/Precision approach deviation indicators for Size A 
photo 


EASE OF INTERPRETING ILS/PRECISION APPROACH INDICATORS SIZE A 
PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 8 Pilot 4: 7 ^ = 7.83 

Pilot 2: 8 Pilot 5:8 q = 0.75 

Pilot 3: 9 Pilot 6: 7 

32. Evaluate the ease of interpreting the ILS/Precision approach deviation indicators Size A 
generic 


EASE OF INTERPRETING ILS/PRECISION APPROACH INDICATORS SIZE A 
GENERIC 


2 

3 

4 

5 

6 

7 

OO 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:8 Pilot 4: 7 % = 7.83 

Pilot 2: 8 Pilot 5:8 ct = 0.75 

Pilot 3: 9 Pilot 6: 7 
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33. Evaluate the ease of predicting flight path using the “follow-me aircraft” for Size A photo: 


EASE OF PREDICTING FLIGHT PATH INFORMATION SIZE A PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 Pilot 4: 8 % = 7.33 

Pilot 2: 7 Pilot 5: 7 c=1.03 

Pilot 3: 9 Pilot 6: 7 

34. Evaluate the ease of predicting flight path using the “follow-me aircraft” for Size A generic: 


EASE OF PREDICTING FLIGHT PATH INFORMATION SIZE A GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 Pilot 4: 8 % = 7.33 

Pilot 2: 7 Pilot 5: 7 c=1.03 

Pilot 3: 9 Pilot 6: 7 

35. Evaluate the ease of using the tunnel for vertical flight path guidance for Size A photo: 


EASE OF USING TUNNEL FOR VERTICAL PATH GUIDANCE SIZE A PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 5 Pilot 4: 7 % = 6. 16 

Pilot 2: 7 Pilot 5: 8 c = 1.47 

Pilot 3: 4 Pilot 6: 6 

36. Evaluate the ease of using the tunnel for vertical flight path guidance for Size A generic: 


EASE OF USING TUNNEL FOR VERTICAL PATH GUIDANCE SIZE A GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 5 Pilot 4: 6 % = 6.33 

Pilot 2: 7 Pilot 5: 8 a =1.21 

Pilot 3: 5 Pilot 6: 7 
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37. Evaluate the ease of using the tunnel for lateral flight path guidance for Size A photo: 


EASE OF USING TUNNEL FOR LATERAL PATH GUIDANCE SIZE A PHOTO 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard 


Somewhat Hard 


Somewhat Easy 


Very Easy 


Pilot 1 : 5 Pilot 4:7 % = 6.0 

Pilot 2: 7 Pilot 5: 7 a =1.26 

Pilot 3: 4 Pilot 6: 6 

38. Evaluate the ease of using the tunnel for lateral flight path guidance for Size A generic: 


EASE OF USING TUNNEL FOR LATERAL PATH GUIDANCE SIZE A GENERIC 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard 


Somewhat Hard 


Somewhat Easy 


Very Easy 


Pilot 1 : 5 Pilot 4: 7 % = 6. 16 

Pilot 2: 7 Pilot 5:7 & = 0.98 

Pilot 3: 5 Pilot 6: 6 

39. Please rate the workload associated between the two different missed approaches using the 
NASA Size A photo-texture display concept. We can define workload as: “the degree of 
cognitive processing capacity required to perform the flight task approach adequately”. Runway 
25: 


WORKLOAD TO RWY. 25 SIZE A PHOTO 


2 

3 

4 

5 

6 

7 

OO 


Very High Somewhat High Somewhat Low Very Low 


Pilot 1 : 5 Pilot 4:8 % = 6.16 

Pilot 2: 3 Pilot 5:9 cr = 2.22 

Pilot 3: 5 Pilot 6: 7 

40. Please rate the workload associated between the two different missed approaches using the 
NASA Size A photo-texture display concept. We can define workload as: “the degree of 
cognitive processing capacity required to perform the flight task approach adequately”. Runway 
7: 


WORKLOAD TO RWY. 7 SIZE A PHOTO 



6 7 8 

9 


Very High Somewhat High 

Somewhat Low 

Very Low 


Pilot 1: 5 

Pilot 4: 6 

x= 

6.0 

Pilot 2: 3 

Pilot 5: 9 

<j= 

2.0 

Pilot 3: 7 

Pilot 6: 6 
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41. Please rate the workload associated between the two different missed approaches using the 
NASA Size A generic-texture display concept. Runway 25: 


WORKLOAD TO RWY. 25 SIZE A GENERIC 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very High 


Somewhat High 


Somewhat Low 


Very Low 


Pilot 1 : 5 Pilot 4:8 % = 6.0 

Pilot 2: 3 Pilot 5:8 a = 2.0 

Pilot 3: 7 Pilot 6: 5 

42. Please rate the workload associated between the two different missed approaches using the 
NASA Size A generic-texture display concept. Runway 7: 


WORKLOAD TO RWY. 7 SIZE A GENERIC 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very High 


Somewhat High 


Somewhat Low 


Very Low 


Pilot 1: 5 Pilot 4: 6 % = 5.67 

Pilot 2: 3 Pilot 5: 8 <j= U5 

Pilot 3: 7 Pilot 6: 5 

43. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 25 for Size A photo. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” 


SA TO RWY 25 SIZE A PHOTO 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1: 7 
Pilot 2: 8 
Pilot 3: 7 


Pilot 4: 9 
Pilot 5: 9 
Pilot 6: 7 


Z =7.83 
< 7 = 0.98 
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44. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 25 for Size A generic. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” 


SA TO RWY. 25 SIZE A GENERIC 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1: 5 Pilot 4: 8 6.83 

Pilot 2: 8 Pilot 5: 6 <j= 1.16 

Pilot 3: 7 Pilot 6: 7 

45. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 7 for Size A photo. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” 


SA TO RWY 7 SIZE A PHOTO 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1: 7 Pilot 4: 9 % = 7.83 

Pilot 2: 8 Pilot 5: 9 &= 0.98 

Pilot 3: 7 Pilot 6: 7 

46. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 7 for Size A generic. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” 


SA TO RWY. 7 SIZE A GENERIC 


2 

3 

4 

5 

6 

7 

OO 


Very Low Somewhat Low Somewhat High Very High 


Pilot 1: 6 
Pilot 2: 8 
Pilot 3: 7 


Pilot 4: 8 
Pilot 5: 6 
Pilot 6: 7 


Z =7.0 
a= 0.89 
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Size X Display Concept Evaluation 


47. Evaluate the ease of interpreting airspeed information for Size X photo: 


EASE OF INTERPRETING AIRSPEED INFORMATION SIZE X PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


X = 8.0 
a = 1.26 

48. Evaluate the ease of interpreting airspeed information for Size X generic: 


Pilot 1: 6 
Pilot 2: 8 
Pilot 3: 9 


Pilot 4: 9 
Pilot 5: 9 
Pilot 6: 7 


EASE OF INTERPRETING AIRSPEED INFORMATION SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


% - 8.0 

a = 1.26 

49. Evaluate the ease of interpreting flight path vectors for Size X photo: 


Pilot 1: 6 
Pilot 2: 8 
Pilot 3: 9 


Pilot 4: 9 
Pilot 5: 9 
Pilot 6: 7 


EASE OF INTERPRETING FLIGHT PATH VECTORS SIZE X PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:6 Pilot 4: 7 ^ = 7.66 

Pilot 2: 8 Pilot 5: 9 a =1.21 

Pilot 3: 9 Pilot 6: 7 

50. Evaluate the ease of interpreting flight path vectors for Size X generic: 


EASE OF INTERPRETING FLIGHT PATH VECTORS SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

OO 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 Pilot 4: 7 % = 7.5 

Pilot 2: 7 Pilot 5: 9 a =1.22 

Pilot 3: 9 Pilot 6: 7 
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5 1 . Evaluate the ease of interpreting altitude information for Size X photo: 


EASE OF INTERPRETING ALTITUDE INFORMATION SIZE X PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 Pilot 4: 8 x = 7.83 

Pilot 2: 7 Pilot 5:9 c = 1 . 17 

Pilot 3: 9 Pilot 6: 8 

52. Evaluate the ease of interpreting altitude information for Size X generic: 


EASE OF INTERPRETING ALTITUDE INFORMATION SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 
Pilot 2: 8 
Pilot 3: 9 


Pilot 4: 8 
Pilot 5: 9 
Pilot 6: 8 


X = 8.0 
a = 1.09 


53. Evaluate the ease of interpreting vertical speed information for Size X photo: 


EASE OF INTERPRETING VERTICAL SPEED INFORMATION SIZE X PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 Pilot 4: 8 x = 7.66 

Pilot 2: 7 Pilot 5: 9 c=1.21 

Pilot 3: 9 Pilot 6: 7 

54. Evaluate the ease of interpreting vertical speed information for Size X generic: 


EASE OF INTERPRETING VERTICAL SPEED INFORMATION SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 
Pilot 2: 8 
Pilot 3: 9 


Pilot 4: 8 x = 7.83 

Pilot 5: 9 a =1.17 

Pilot 6: 7 
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55. Evaluate the ease of interpreting the ILS/Precision approach deviation indicators for Size X 
photo 


EASE OF INTERPRETING ILS/PRECISION APPROACH INDICATORS SIZE X 
PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 Pilot 4: 9 ^ = 7.83 

Pilot 2: 8 Pilot 5: 8 c=1.17 

Pilot 3: 9 Pilot 6: 7 

56. Evaluate the ease of interpreting the ILS/Precision approach deviation indicators Size X 
generic 


EASE OF INTERPRETING ILS/PRECISION APPROACH INDICATORS SIZE X 
GENERIC 


1 2 3 4 5 6 7 8 9 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:6 Pilot 4: 9 ^ = 7.83 

Pilot 2: 8 Pilot 5:8 0=1.17 

Pilot 3: 9 Pilot 6: 7 

57. Evaluate the ease of predicting flight path using the “follow-me aircraft” for Size X photo: 


EASE OF PREDICTING FLIGHT PATH INFORMATION SIZE X PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 7 Pilot 4: 7 % = 5.83 

Pilot 2: 7 Pilot 5:8 q - 2.79 

Pilot 3: 4 Pilot 6: 8 

58. Evaluate the ease of predicting flight path using the “follow-me aircraft” for Size X generic: 


EASE OF PREDICTING FLIGHT PATH INFORMATION SIZE X GENERIC 


1 2 3 4 5 6 7 8 9 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 7 
Pilot 2: 7 
Pilot 3: 4 


Pilot 4: 7 
Pilot 5: 8 
Pilot 6: 8 


X = 6.83 
0= 1.47 
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59. Evaluate the ease of using the tunnel for vertical flight path guidance for Size X photo: 


EASE OF USING TUNNEL FOR VERTICAL PATH GUIDANCE SIZE X PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:6 Pilot 4: 9 ^ = 6.67 

Pilot 2: 7 Pilot 5:8 q - 2.50 

Pilot 3: 7 Pilot 6: 8 

60. Evaluate the ease of using the tunnel for vertical flight path guidance for Size X generic: 


EASE OF USING TUNNEL FOR VERTICAL PATH GUIDANCE SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1:6 Pilot 4: 9 ^ = 6.67 

Pilot 2: 7 Pilot 5:8 a - 2.50 

Pilot 3: 7 Pilot 6: 8 

6 1 . Evaluate the ease of using the tunnel for lateral flight path guidance for Size X photo: 


EASE OF USING TUNNEL FOR LATERAL PATH GUIDANCE SIZE X PHOTO 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 Pilot 4: 9 ^ = 6.5 

Pilot 2: 7 Pilot 5:7 q = 2.43 

Pilot 3: 7 Pilot 6: 8 

62. Evaluate the ease of using the tunnel for lateral flight path guidance for Size X generic: 


EASE OF USING TUNNEL FOR LATERAL PATH GUIDANCE SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Hard Somewhat Hard Somewhat Easy Very Easy 


Pilot 1: 6 
Pilot 2: 7 
Pilot 3: 7 


Pilot 4: 9 i = 6.0 

Pilot 5:7 a = 2.60 

Pilot 6: 8 
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63. Please rate the workload associated between the two different missed approaches using the 
NASA Size X photo-texture display concept. We can define workload as: “the degree of 
cognitive processing capacity required to perform the flight task approach adequately” Runway 
25: 


WORKLOAD TO RWY. 25 SIZE X PHOTO 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very High 


Somewhat High 


Somewhat Low 


Very Low 


Pilot 1: 5 Pilot 4: 8 % = 6.0 

Pilot 2: 4 Pilot 5: 7 o=1.54 

Pilot 3: 7 Pilot 6: 7 

64. Please rate the workload associated between the two different missed approaches using the 
NASA Size X photo-texture display concept. We can define workload as: “the degree of 
cognitive processing capacity required to perform the flight task approach adequately” Runway 7 : 


WORKLOAD TO RWY. 7 SIZE X PHOTO 


6 7 8 

9 

Very High Somewhat High 

Somewhat Low 

Very Low 


Pilot 1: 5 Pilot 4: 7 % = 6.16 

Pilot 2: 4 Pilot 5: 7 o=1.33 

Pilot 3: 7 Pilot 6: 7 


65. Please rate the workload associated between the two different missed approaches using the 
NASA Size X generic-texture display concept. Runway 25: 


WORKLOAD TO RWY. 25 SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very High Somewhat High Somewhat Low Very Low 


Pilot 1 : 5 Pilot 4: 8 % = 6.33 

Pilot 2: 5 Pilot 5: 6 &= 1,21 

Pilot 3: 7 Pilot 6: 7 
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66. Please rate the workload associated between the two different missed approaches using the 
NASA Size X generic-texture display concept. Runway 7: 


WORKLOAD TO RWY. 7 SIZE X GENERIC 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very High 


Somewhat High 


Somewhat Low 


Very Low 


Pilot 1 : 5 Pilot 4:7 % = 6.33 

Pilot 2: 5 Pilot 5: 7 &= / 03 

Pilot 3: 7 Pilot 6: 7 

67. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 25 for each display concept. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” Size X Photo-texture: 


SA TO RWY 25 SIZE X PHOTO 


1 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1 : 3 Pilot 4: 9 % = 7.5 

Pilot 2: 8 Pilot 5: 9 <7=2.34 

Pilot 3: 9 Pilot 6: 7 

68. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 25 for each display concept. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” Size X Generic -texture: 


SA TO RWY. 25 SIZE X GENERIC 


1 2 3 4 5 6 7 8 9 


Very Low Somewhat Low Somewhat High Very High 


Pilot 1: 4 

Pilot 4: 8 

Z =7.33 

Pilot 2: 8 

Pilot 5: 9 

<7=1.97 

Pilot 3: 9 

Pilot 6: 6 
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69. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 7 for each display concept. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” Size X Photo-texture: 


SA TO RWY 7 SIZE X PHOTO 


1 


2 

3 

4 

5 

6 

7 

8 | 


Very Low 


Somewhat Low 


Somewhat High 


Very High 


Pilot 1 : 3 Pilot 4:9 j = 7.33 

Pilot 2: 7 Pilot 5: 9 <j= 2.33 

Pilot 3: 9 Pilot 6: 7 

70. Please evaluate the level of situation awareness experienced during the missed approach to 
rwy. 7 for each display concept. We define situation awareness as: “...the pilot has an integrated 
understanding of the factors that will contribute to the safe flying of the aircraft under normal or 
non-normal conditions.” Size X Generic -texture: 


SA TO RWY. 7 SIZE X GENERIC 


2 

3 

4 

5 

6 

7 

8 | 


Very Low Somewhat Low Somewhat High Very High 


Pilot 1: 4 

Pilot 4: 8 

%= 7.33 

Pilot 2: 7 

Pilot 5: 9 

( 7 = 1.86 

Pilot 3: 9 

Pilot 6: 7 
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